It started bugging me that different coin amounts were being awarded at the end of a song on a seemingly random basis, so I tried to figure out how coins are awarded through some experimentation. I haven't confirmed every single scenario, but this formula should be correct if it applies across the board like I think it does:

Band Tiers 0-1 have a base amount of 50 coins.
Band Tiers 2-4 have a base amount of 75 coins.
Band Tiers 5-6 have a base amount of 100 coins.

To calculate how many coins you will be awarded for playing a song, check the band tier and you will have the base amount of coins as listed above. The base amount is then multiplied by how many stars you get (0-6, with 6 being gold stars). Currently, playing new songs doubles the amount of coins you get.

So for an example of calculating how many coins you should earn, if you play a Tier 4 song, get 5 stars, and you're playing that song for the first time, you can expect to get 75 x 5 x 2 = 750 coins. If you used all of your power-up slots for the song (which total up to 750 spent coins), you would essentially break even.

Using all three power-up slots on Tier 0-1 songs is generally going to be bad for your overall coin count.

Hope this info is correct and that other people find it useful!