How do you determine the cost of an online ad?

Considering the current advertising climate, unfortunately, the price isn't going to be as high as it once might have been. Due to the dot-com crash, and because the content site market is so saturated, CPM rates have taken a severe dive recently. If you have a quality site, however, it's still possible to sell enough advertising to keep yourself afloat. Keep in mind that the asking price, on your rate card, will almost certainly not be what you wind up charging the advertiser, as nine times out of ten they'll negotiate the rate down. Remember, you aren't trying to milk advertisers for all they're worth; rather you're looking to build a long-lasting relationship that ensures that they buy more than once.

It is a competitive market so the best thing you can do is look on your competitor sites and see what they charge thats what your prospective buyers will do anyway. Make sure you try to compare apples to apples to the extent you can - same PR, Alexa rank, target market etc.

Below is a tool that should be helpful to you it can be found at:

Adsense Calculator - SEO Tools - Search Engine Optimization, Google Optimization

SEO Tools - Adsense Calculator

The AdSense Calculator helps Google AdSense affiliates understand what affects their earnings by letting them experiment with values.
It also allows Google AdSense users to take their current statistics and calculate how much they can expect to earn daily, monthly and yearly. Site owners who are considering implementing AdSense on their site(s) can also use this tool to estimate what results they are likely to see.

Daily Page Impressions
Daily page impressions indicate how many times ads are shown per page. You can use your average, anticipated, or current daily page impressions for this

Click Through Rate (CTR)
This figure represents how many visitors actually click your advertisements. You can find this easily on your Google AdSense stats page labeled "CTR." The average rate is estimated to be about 1.5%.

Cost Per Click (CPC)
Your CPC is the average of how much you get paid for someone clicking one of your advertisements. To find your average CPC take you’re the amount of money you have earned from AdSense (daily, monthly, or all time) and divide it by your total clicks (daily, monthly, or all time).

Thanks for the link!!!

I think you will fine this tool to be helpful. It will help you determine the approximate amount you should be paying (or charging) per month for a text link (ad) from each and every page of the specified website.

It takes into consideration factors such as number of Backlinks, Alexa traffic rank, age of the website, etc.

Link Price Calculator

I agree with TXA the best way you can determine the price of ads is by finding out what your competitors are charging!