The red one closes the window, but does not quit the app. The yellow one minimizes it to the dock. Use that, and you won't have any problem.