How to Get Detailed PPC Keyword Data from Google Analytics
Google Analytics, Hacks and Tools Add comments
At GA-Experts, we also provide PPC Management services through our parent company Omega Digital Media, so we are continually looking at ways to improve the data that Google Analytics provides about a PPC campaign. With this in mind we have developed a new filter that shows exactly the Search Term (i.e. keywords that are searched on) rather than just the Bid Term (i.e. the keywords that triggered a PPC advert).
For example, if a visitor searches for “sony digital camera” and you have an ad configured for the keywords “digital camera” as a Broad Match in Google AdWords, then this search will show your ad. But in your GA reports, the keywords for this search will be reported as “digital camera“. Whilst this gives you good data on your Bid Term, it doesn’t help you refine your keyword triggers because the reports don’t show the actual Search Term.
If you use this Override Bid Term filter then the same search will show “digital camera, (sony+digital+camera)” in any Google Analytics report that shows keywords (e.g. Marketing Optimisation -> Search Engine Marketing -> Overall Keyword Conversion) or any time you cross-segment against keywords.
This data now allows you to fine-tune your PPC campaigns by getting greater detail from your GA reports. In the example above, if the ‘sony+digital+camera‘ search is a frequent one, you might consider creating an Exact Match advert that took visitors to a dedicated Sony landing page. Or replacing the Broad Match ‘digital camera‘ term with a more precise ‘sony digital camera‘ Exact Match to increase your clickthrough rate.
There’s lots that can be done to optimise a campaign using this filter, so please see below for configuration instructions:
1) Create two filters as per the images here:
The Field A regular expression is: (\?|&)(q|p)=([^&]*)
In English, this means:
Look for character immediately following a ? or & (i.e. a URL variable), that is named q or p, and extract the value of this variable.
2) Create a new profile for your existing site. This keeps the detailed keyword data in a separate profile. You can apply this filter to your main profile(s) if you prefer, but a separate profile allows for continuity and top level reporting in your main profile with keyword detail available as required.
3) Apply the two filters to the new profile, in the order Override Bid Term 1 *then* Override Bid Term 2
You can now view detailed Search Term data from any GA report that shows keyword information. Please let us know any feedback via the blog comments.
For further examples of using filters, get a copy of the book Advanced Web Metrics with Google Analytics published by Wiley & Sons, March 2008 (ISBN: 978-0-470-25312-0).
November 3rd, 2006 at 9:18 am
Jim,
This is *brilliant*. Bravo!
Timothy
November 17th, 2006 at 2:44 pm
This is so lovely. I read this a week or more ago and finally had time to implement for a customer. Thank you for the screen shots, they make it trivial.
Robbin Steif
LunaMetrics
December 7th, 2006 at 11:08 am
This is great stuff guys. I really like the referrals post too. I have been looking for these stats for a while. Previously I had to take the strain of outputting it through burny raw log files.
This really takes GA up to another level. The extra keywords keywords are great for refining adwords campaigns and I am about to setup an affiliate campaign for a client, so looking at the referrals from the Google content network is a great way for targeting websites who are commercial aware and could potentially be great affiliates.
Keep up the good work…
If only I knew how to create a filter so I could see stats for Robots/Spiders …hmmm…
December 7th, 2006 at 11:39 am
Thanks Joe. Just a quick point - you won’t see search engines or spiders in GA reports as in order to be logged by GA as a site visitor you need to process the JavaScript page tags. At the moment, despite rumours to the contrary, we don’t believe there are any SE spiders that can process JavaScript (processing power, security issues, etc.) - but in the future who knows…
January 10th, 2007 at 7:09 pm
hello there. thanks so much for the helpful tips on this. Is it possible to ask one question?
in the ‘override bid term 1′ filter the output to constructor is a custom field 1 with a varible called $A3.
in the override bid term 2 filter the output to constructor is a campaign term with the regex of: $B1,($A1). Is this a typo or should the regex be: $B1,($A3) to match the variable in filter 1 above?
sorry for the question. i am new to this.
appreciate it very much!
jay wilner - out of bounds
January 10th, 2007 at 7:10 pm
thanks for the help on this. one question. in the override bid term 1 filter, you have the output to-> going to $A3. Is this supposed to be $A1 to match that term in Override Bid term 2 constructor? just checking, i dont know this stuff too well. or do we just follow exactly like you have it? thanks again!! jay wilner - out of bounds
January 10th, 2007 at 7:11 pm
Great writeup guys! So many analytics available to us now, and its good when we’re able to combine processes to drill down into the real value of stats.
April 2nd, 2007 at 4:42 pm
Sorry this doesnt work for me, Ive put the code of every page of my site (for the new profile) - analytivs is showing as receiving data - but theres nothing in my reports not even a single visitor for the last 4 days which is totally wrong.
can anyone help me?
thanks in advance
April 2nd, 2007 at 4:42 pm
The filter is correct - and working for this site.
$A3 relates to the matching in the regular expression. It is not stored in the filter.
HTH
August 18th, 2007 at 12:43 am
[...] GA-Experts » Blog Archive » How to Get Detailed PPC Keyword Data from Google Analytics (tags: google+analytics) [...]
August 30th, 2007 at 10:22 pm
This is really great. Thank you for taking the time to share it!
October 29th, 2007 at 11:06 pm
I’ve been looking for this kind of solution for a long time. It only took a couple minutes to set-up and the detailed keyword tracking is perfect. The data on broad matches are already showing areas that need to be adjusted. By the way, the link from ROI Revolution is linking to a 404ed page.
Thanks for the filter.
Adam
November 2nd, 2007 at 3:55 am
I installed the filters and they work great! Only problem is that they don’t appear to be able to track conversions. Anyone have any luck figuring that out?
Great hack none the less.
November 3rd, 2007 at 3:56 pm
Hi,
Great tip, but I can’t get it to work! I installed and applied the filters, but can’t see the ‘raw’ search terms … (Eg. in ‘Traffice Sources -> Keywords’) Do i just have to be patience or am I doing something wrong?
Thanks, Bas.
November 3rd, 2007 at 10:27 pm
Never mind, patience is not my thing I suppose ..
It works great, just what I wanted to see! Thanks!
January 28th, 2008 at 1:23 am
I tried the code above (double checked everything) but I am not getting the detail information. I suspect similar problem as to Ingo’s - we are also running on the new GA code… Anybody else having problems with this?
January 28th, 2008 at 8:49 pm
[...] this secret to our client accounts and our own accounts comes from a November 2007 blog post by the GA-Experts. Thank you, GA-Experts, for sharing this secret with [...]
February 4th, 2008 at 9:09 pm
Like Paul, I can’t seem to get this to work. Have you identified any account configuration issues that might explain the problems we are having? Thanks for publishing the technique! If I can get it working it will be very powerful. — Jaimie Scott
February 11th, 2008 at 12:31 pm
We’re seeing exactly the same thing happening. We have this code running on 10 of our clients accounts and all the data has stopped being reported on all the accounts.
We’re running the old .js code on these accounts.
February 14th, 2008 at 7:20 pm
Same thing here…had this filter working on several client accounts and then Jan. 8th hit and it no longer works. Something definitely happened that day…
February 14th, 2008 at 7:45 pm
I have heard that Custom Field 1, which is used in this filter, became read-only on January 8th. Looks like this filter may be out of commission.
February 18th, 2008 at 7:53 am
Hi Tyson,
yes, the same here. The filter worked without any problems until Jan 8th. After that all data turned into “not set”.
Do you have any statement from Google that there was a change on custom field?
February 19th, 2008 at 7:17 pm
Anyone know if this can be fixed?
February 21st, 2008 at 10:59 am
This is the response I received from Google so far:
“The filters are not officially supported by Google. That’s why we cannot offer any support here.
Please check whether the auto tagging function in Google AdWords is enabled and that cost data is applied to the Google Analytics profile.”
That hasn’t solved my problem since both check boxes are checked. It would be good to get in touch with somebody where the filters still work in order to compare settings. Any volunteers?
May 31st, 2008 at 7:18 am
Here’s the fix:
http://www.roirevolution.com/blog/2007/04/exact_keyword_tracking_with_google_analytics_revis.html
November 5th, 2008 at 12:27 pm
[...] Advance PPC data filter: This is a link to probably the best filter tutorial that I have ever seen! This filter allows you [...]
November 6th, 2008 at 10:45 pm
Guys, I am not sure what the issue has been here (not working in Jan etc), but I have just implemented these filters for a client that uses ga.js and all is working nicely.
November 6th, 2008 at 11:43 pm
ROI’s version is a slightly different take on the issue (see comment #25).
Essentially they use JavaScript code to set a persistant cookie (utmv) on the visitor’s browser. That way the original search term stays with the visitor on subsequent visits.
My hack is on a per session basis i.e. if the visitor comes back the new search term is captured and reported on next to the bid term.
HTH, Brian
November 8th, 2008 at 3:03 pm
The filter works great! Is there any way to know whether the Bid Term that matched was the Exact, phrase or broad term?
I have very different conversion rates between the Phrase and Exact, but both are shown in GA as ‘My Term’ on 2 rows with no indication of broad or phrase.
A solution for this would also be useful.
November 11th, 2008 at 9:36 pm
I have followed the tutorial and double checked my settings but it still does not seem to work for me. I have tried with Custom Field 1, Custom Field 2, User Defined etc. Can someone tell me if this filter does still work? or should I look to use the ‘ga_keyword.js’ script instead of filters. Any help is much appreciated.
November 13th, 2008 at 8:42 am
Andrew - all works well with the filter. See last two comments
November 15th, 2008 at 4:23 pm
I’ve just been setting up these filters on a new site for the first time in a while and have noticed a couple of things.
1. Minor change to the terms used in the new version of GA. “Custom Field 1″ now seems to tbe “Customised Field 1″..
but also
2. Although the scripts certainly seem to still work, the effects seem slightly different to what I had expected or remembered from the past. For each key phrase I’m seeing two versions listed in the ‘paid’ keywords report
word1 word2 (word1+word2)
word1 word2
The visits figure is split between the two rows. Comparing with an unfiltered profile the combined total is slightly out, but reasonably close.
The e-commerce transactions are all associated with the word1 word2 row.
I was not surprised to see that the ecommerce transactions were not associated with the manipulated version of the data. I think that was also the case when I had used these filters on other sites in the past, or had done similar things such as testing for the existence of some generic emailcode tag to manipulate the ‘medium’ of a visit.
But I was surprised by the existence of the two rows. I would have expected the filter to repeat the actual phrase typed even if it was the same as the one which would have normally been displayed and for there to be no rows without the words in brackets. (I’ve not explained that very well…)
Did these filters always produce results like this? I don’t have my old examples to go back to unfortunately, so cannot check if I am remembering it wrong.
Also: what do the two rows represent, please? The ‘naked’ version is not simply recording the visits which have the transactions. Plenty of these visits do not convert. Are some of them cpc visits which only had partial referral information for some other reason (in my test I have drilled down and, conveniently, all came from Google) I wonder?
I still think these are extremely useful filters and would just like a bit more insight into what may be going on here.
Tim
November 17th, 2008 at 11:11 pm
[...] in November of 2006, our friends at GA Experts posted a method to get detailed PPC keyword data from Google Analytics. This method involved creating a seperate profile and using some intimidating [...]
November 25th, 2008 at 12:24 am
Knowing the exact keywords people typed and then clicked on your ads is extremelly useful. I don’t know why Google Analytics doesn’t show those keywords instead of the broad match ones. I wrote about this not long time back, Best PPC Practice - Exact Keywords.
December 1st, 2008 at 3:17 pm
Tim,
I have the exact same issue in all respects and it would be helpful to know the answers…
December 3rd, 2008 at 3:37 am
I have spotted the same issues with Tim. And there is also something else.
All my keywords with (word1+word2 +/or something else) variations reported by the filter have dramatic differences in pages/visit compared to every other keywords without variations (reported by the filter).
E.x. 1-2 pages/visit for all keywords with variations against 16+ pages/visit for keywords without variations
I believe that this is not justified because the avg time on site remains about the same for both occasions
December 22nd, 2008 at 9:28 pm
Is there any way to show the conversions for the actual “searched” keywords in Analytics. It looks as if Analytics only shows conversions for the keyword triggered not the actual words used for the search.
December 31st, 2008 at 1:26 pm
Great article, very useful!
I have also noticed the same issues as Tim which are..
1. There are duplicate entries for keywords in:
Traffic Sources -> Keywords -> Show: Paid -> Ecommerce
For example:
-My Keyword
-My Keyword, (My+Keyword)
The data for each is different. I understand that the second one is created by the filter, but shouldn’t the visits be the same? Can anyone explain the difference between these rows?
2. In the same Ecommerce section, keywords with (My+Keyword) values do not show any Transaction data. Is there a way to get (My+Keyword) rows to display the correct transaction data?
Note: I noticed that if you filter by “Dimension: User Defined Value”, the keywords appear to display the correct data but I’m not 100% sure. Perhaps there is a way to copy this data to its (My+Keyword) equal? Just a thought
Thanks for any help!
-Joe
January 3rd, 2009 at 8:35 pm
[...] articles give some useful tools for doing this (plus give a good description of the issues): How to Get Detailed PPC Keyword Data from Google Analytics | Google Analytics Experts and Exact Keyword Tracking with ga.js: Unofficial Google Analytics Blog I use the first approach [...]