I also build a smaller 24″ x 24″ frame and started using a Dremel. The Dremel worked well for cutting foam board and MDF, but it didn’t work very well for acrylic. Acrylic heats up and melts so you need to slow your spindle speed or increase your feed rate. The Dremel didn’t have enough power to handle either. I had better luck with my $30 Harbor Freight trim router. I could slow down the spindle speed and increase the feed rate, plus I could find single flute bits.
Here are a few tips I found that may help. Mix some dish soap with water and put it on the acrylic. I also put some in a spray bottle so it can be sprayed on when it’s running. This cools and helps lubricate the shavings so they don’t tend to melt back together. Clean the shaving out of any cuts if you can so they don’t melt and remove the acrylic when it starts building up on the bit. Do several shallower cuts instead of trying to go too deep. I was cutting thinner extruded acrylic, but I read that cast acrylic cuts better. I never tried cutting cast.
One more thing that may be interesting to try is using compressed air to cool and blow away the shavings.