Update 2: Hurray, it finally works.
Empirically I found out that the mp3 file format causes the problem in FF (no controls shown). Adding an ogg file as additional source did the trick!
This works also in a normal page, it doesn’t have to be a audio format post.
I didn’t manage to get it via the attachment function, but I had success by editing the HTML tab like this (Replace the brackets with html tag symbols):
[p][audio][br /]
[source src=”path.mp3″][br /]
[source src=”path.ogg”][br /]
[/audio][/p]
Be careful. Switching back to the visual tab may destroy the html code. This is bloody annoying…