Stumper: Brake booster failing

Got a braking problem that has me stumped:
My booster seems to take time to get up to vacuum - first stop is fine but if I hit the brakes again, no boost - I have to really stomp on it.
After 5- 10 minutes it's back to working - for one stop tha is...

I'm on my third booster and the problem is still there.
I borrowed a vacuum gage from Auto Zone and the vacuum off the manifold is good (around 15 as I recall), just takes a while to get there.

The booster holds vacuum and will drop to roughly half when you press the pedal down (you can hear it suck the air in under the dash with the engine off). Don't know any other tests, so IMO it's working fine.

I am running a tow cam, but it's been 2 years since the rebuild and think the cam isn't the problem.

I did put a canister on it (no difference), checked the check valve (working fine) and replaced the hoses with new vacuum lines...

Could it be the carb (8 year old AFB)? valve adjustment? timing?
[addsig]