I am reading a very interesting paper by Thomas Blake, Chris Nosko, and Steven Tadelis about eBay and search engine marketing. They find through an experimental design that there is virtually no significant effect on eBay sales when they stop paid keyword search placements. One of the critical factors was using Google’s ability to tweak keywords for different geographies. They find that while new or infrequent users are drawn in by the search ads, their sales are minimal compared to the swath of repeat customers who don't need the ads to direct them to eBay. There is a discussion of implications about the (lack of) value of search engine marketing for large, established companies in the paper.
But startups are far from your typical large, established companies. I’m not sure how effective search engine marketing is for startups, and this paper doesn’t speak to that audience. What this paper does offer is some general evidence about organic versus paid search traffic, which is relevant to startups, particularly if one becomes more widely known. And it critically cautions against the pitfalls of a straightforward return on investment calculation (bold is my emphasis):
Not only do most consulting firms who provide marketing analytics services use observational data, recommendations from Google offer analytical advice that is not consistent with true causal estimates of ad effectiveness. As an example, consider the advice that Google offers its customers to calculate ROI:
“Determining your AdWords ROI can be a very straightforward process if your
business goal is web-based sales. You’ll already have the advertising costs for
a specific time period for your AdWords account in the statistics from your
Campaigns tab. The net profit for your business can then be calculated based
on your company’s revenue from sales made via your AdWords advertising,
minus the cost of your advertising. Divide your net profit by the advertising
costs to get your AdWords ROI for that time period.” [a footnote cites this quote to here].
This advice is very much akin to running a regression of sales on adwords [sic] expenditures, which is not too different from the approach that result in the inflated ROIs that we report in columns 1 and 2 of Table 5. It does not, however account for the endogeneity concern that our study highlights where consumers who use paid search advertising on their way to a purchase may have completed that purchase even without the paid search ads.
There’s nothing stopping from you conducting your own experiment if you wanted to go down that route.