I think 4-6 might be a tad low for performance. Even though the VZW TP has 96 MB less RAM than the Sprint version (192MB vs. 288MB), I don't think I'd go down to 6 or lower. I think it might be laggy if set that low. 6MB might be okay for a stock ROM, but a WM6.5 ROM is going to need a bit more breathing room.
I dunno, maybe 6-12MB somewhere?