tldr: They seem to be the same species. D. musaica is the valid name.
I compared the references given for each species listed on ipni.org. They match very closely - closely enough to be the same species.
The description given for D. reginae is significantly more detailed, so it contained additional information about various things like the purple flush on immature plants and more details about the flowers. The mosaica description was written in German (in the 1800s). This led to some odd translations- most notably the use of the word hellbraun in describing the colour of the leaf undersides and ovaries. According to google translate the translates to light brown or tan. However, the same word was used to describe the leaves of Geogenanthus poepeggi which are a similar purple shade. I can't find any information that specifically talks about this but it seems like its just a translation that may have changed over time.
Everything that was on both descriptions matched other than the width of the leaves. It seems, given everything else being identical, this would just be natural variation within the species.
I also had a look at powo and they cite a 2008 paper for their source as to why accept musaica over reginae. The main reason the paper gives is that the publication for D. reginae is considered invalid as they did not cite the date of publication for the basionym.
Also, most importantly, I reached out to wfo and spoke to one of their commelinaceae experts who confirmed that it was an oversight when the site moved over from the plant list (this happened a couple years ago). He said he would update it so hopefully it will be labelled a synonym on the next site update, on the summer solstice (I will check).
Site links if anyone wants to check it out for themselves (I was gonna link the papers to but I closed all the tabs and can't find them sorry but found them all free on google):
wfo - https://wfoplantlist.org/
ipni - https://www.ipni.org/
powo - https://powo.science.kew.org