"Does it happen with both the front brake lever and foot switch or just one?"
That's a VERY good question from a diagnostic point of view:
My FXSTC Rear Brake (foot) did not light the brake-light whilst the Front Brake (hand) did. It's a common problem with the hydraulic brake switch that uses brake-fluid pressure to light the brake-light when the Rear Brake (foot) is applied.
I replaced the switch and for curiosity i tested the impedence on the old one. In an 'open' condition the resistance should have been infinite, however there was a measureable impedence indicating that the switch was not completely 'open'. Thus some current was flowing. Perhaps, if you discover the situation i did, some current is flowing and thus it lights you tester but the total current is insufficient to illuminate the brake-light filament.
Answer the first question and go from there - diagnostically speaking.