Most importantly, I think you need more of a grit gradient. Rubbing compound is equivalent to a grit of probably 5000 or so. You'd need to work forever to eliminate 400 grit scratches with 5000 grit abrasive.
The typical technique is to use diamond pads, for example I've used these:
Flex diamond pads
but to get a really good finish you can't skip many steps, just as with polishing metal or anything. You probably don't want to spend $400 to get a good new set, however.
Try Ebay. Here's a complete set that is closing in 36 hours and currently on $35:
diamond pads on ebay
Also, diamond is surely not required for marble, given its softness. Someone might make pads like those above from less expensive abrasives (that is, specifially for marble but not granite and harder stones), but I'm not aware of them.
However, you can get aluminum oxide or silicon carbide sheet abrasives for about a buck and up each, and these can be had in discs from 4" to 16" diameter, sheets, belts, with or without adhesive, etc. 3M makes about 5,000 different configurations. I am pretty sure silicon carbide would be fine for marble.
Standard gray wet-or-dry paper is silicon carbide, (I'm assuming that's what you used). The same thing is available up to a grit equivalent of about 100,000 (finer that 600 they grade by micron size, not 'grit') These materials are typically called "Lapping Films". 3M only sells them in bulk packages of 10 or 50 or 100, but there are a few places that will sell individual sheets. Here's where I buy some of mine (that I use on metal):
PSI - Precision Surfaces International
The menu is complicated, but try following this:
Online Store
- Metallography
-- Lapping Film Discs
--- Presuure Sensitive Adhesives
----- [ then pick your size ]
4" discs are $0.40 each from 30 to .3 micron. 30 micron is 600 grit. So you can get wet-or-dry paper from the hardware store up to 600 grit, and then go from there up to sub-micron ranges (.3 micon is like 100,000 grit and will produce optical finish) Realisitically, for marble you probably want to go up to 5 micron max (4000 grit), but this is a guess on my part.
The guy at PSI is nice and helpful, although don't expect him to know about marble, because that's not his customer base. But if you've proven that wet-or-dry paper cuts marble fine, then these lapping films are sure to work because it's tha abrasive, just finer grades.
Good luck!