What am I missing!!!

[Supermario]

As you all know, I'm in the process of polishing sues trailer from Izzy.
We did our homework, got all the grades of sandpaper, pre cleaned the metal with oven cleaner for a 10 minute soak etc.
Heres the issue: I can make the aluminum look like perfect chrome finish as clear as day when you look straight at it. At various angles you can see the swirl marks and it looks like %#*¥#.
If I use mothers after , it gets rid of the swirl lines but the look is foggy and not as crisp. I have tried everything and at a point that no matter what I do, I can't get rid of the swirl and keep the clear look!
So anxious to get this trailer polished , wired and back together. I'm getting frustrated being stuck in the first step still
Here's what I been up to...
I'm using felt wheels on a dryer motor, I have 2 medium felt 1" thick 5" dia wheels stacked together for a 2 inch buffing surface. I use the black with the medium wheel. Then I switch to the same set up but with a soft felt wheel and use the white. Like I said , to look straight on at the plate it looks perfect but can see swirls from an angle. I have a small wire brush and I rake the wheel frequently, I tried one wheel only and not 2 , I tried a brand new soft wheel with no polish and just buff. I'm so close and yet I feel like I'm missing one step.......

Any ideas or suggestions is greatly appreciated !

Mario