In exercising the above-mentioned powers, the Board may reverse or affirm, wholly or partly, or may modify the order, requirement, decision or determination, including any order requiring an alleged violator to stop, cease and desist, and may make such order, requirement, decision or determination, including stop orders or orders to cease and desist as ought to be made. Notice of such decision shall forthwith be given to all parties in interest.