Generating a PWM frequency greater than 125 kHz using Arduino Uno

It's a bit beyond the normal Arduino capabilites so you need to delve into setting some of the registers for your ATMEGA chip directly. See the "Using the ATmega PWM registers directly" http://arduino.cc/en/Tutorial/SecretsOfArduinoPWM

My requirement of 125 kHz is satisfied, but I am just curious to know why the sketch not working as it should.

According to my measurements, it is. Using this slightly modified version of your code:

// A sketch that creates an 8 MHz, 50% duty cycle PWM and a 250 kHz,
// 6-bit resolution PWM with varying duty cycle (changes every 5 µs
// or about every period.

#include <avr/io.h>
#include <util/delay.h>

int main(void)
{
  pinMode(3, OUTPUT); // Output pin for OCR2B
  pinMode(5, OUTPUT); // Output pin for OCR0B

  // Set up the 250 kHz output
  TCCR2A = bit(COM2A1) | bit(COM2B1) | bit(WGM21) | bit(WGM20);
  TCCR2B = bit(WGM22) | bit(CS20);
  OCR2A = 63;
  OCR2B = 31;

  // Set up the 8 MHz output
  TCCR0A = bit(COM0A1) | bit(COM0B1) | bit(WGM01) | bit(WGM00);
  TCCR0B = bit(WGM02) | bit(CS00);
  OCR0A = 1;
  OCR0B = 0;
}

I get the predicted 8 MHz output on pin 5:

And the 250 kHz output on pin 3:

Yes, you can reduce the delay.

From the documentation:

This function works very accurately in the range 3 microseconds and up. We cannot assure that delayMicroseconds will perform precisely for smaller delay-times.

Note that, there are a few cycles between the delays, so as you get shorter timeperiods, you will get less and less accuracy.

You could hand-code the delay. I would imagine you could get up to 1 MHz, but with little control over the duty cycle (at that frequency).You would need to disable interrupts, and you could do nothing else at all.

If you want still more, you could set a fuse on the ATmega, and get your native clock speed out on of pin 14 CKO (this shows up as digital pin 8 on the Arduino board) - I would expect a 50% duty cycle; you will have NO control over - you can't switch it on or off, or affect the duty cycle. It can be used to run multiple chips off the same clock.

To your original question: You can reduce the delay, but there's a limit to how small the delay can be.

Two reasons: Firstly, the loop() function does have some overhead, and secondly the code that you write takes time to execute. This is why the hardware PWM timer is valuable - it generates the pulses without much software intervention.

I used the code below and it resulted in 125 kHz and 1.6 MHz (measured with a CRO, not simulation).

This is really just to add to Nick's answer... (To understand what's going on, you need to refer to the datasheet. Based on your questions, I'm assuming you haven't stared at it long enough.)

In the first case: 125 kHz.

I am not sure if you're aware, but you're using a special mode of "Fast PWM" which is slightly different from the analogWrite() provided in Arduino.

In this mode, your PWM period is determined by the time it takes your timer counter to match the value in OCR2A register. This means that with a 16 MHz clock, and your output is toggled at a rate of 16 MHz/63 = 250 kHz (roughly).

So far so good, but what does the CRO see? The CRO considers a period to be made up of both high and low signal. It takes TWO 250 kHz cycles to toggle the signal up and down. Hence, the reading shows 125 kHz.

As to why the rolling would make your reading difficult: Think what kind of wave will appear when OCR0B = 0. ;-)

Case 2: 8 MHz

The same applies in the seccond case, and you should expect a 16 MHz cycle that toggles the signal up and down (which gives a 8 MHz reading on the CRO). As to why you are getting 1.6 MHz - that's an odd case. Can you post your CRO reading?