Provided you have created your Google API Credentials, you can start to programmatically pull data from Google Search Console. In this article, I’m going to walk you through authorizing your script to prepare us for pulling reports.
Requirements
pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib
Before We Start Coding
Before we get into the script, let’s do a little housekeeping. At the end of the article where we create the Google credentials, I mentioned you should download the OAuth JSON file. In our script, we are going to be opening this file to authorize it. I recommend renaming it to credentials.json
, since that’s what we’re going to be calling in our script. I also recommend moving it to the same folder that you are writing this script in. Your folder structure might look something like this:
- gsc-api
- credentials.json
- gsc-api.py
Authorizing Google Search Console API in Python Script
Let’s start off by mentioning that Google supplies various ways to connect to their API services. I built this by using the Google Sheets API documentation, because it seemed to be the most recent and straight forward. Of course, I modified it a little bit so we are authorizing Search Console instead of Sheets.
We start by importing the modules we’ll be using.
import os.path
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials
Define Authorization Function
Next we will define the function that we’ll need to use to authorize the credentials. As always, I’ll walk you through some of what this function does line by line.
def gsc_auth(scopes):
creds = None
# The file token.json stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token.json'):
creds = Credentials.from_authorized_user_file('token.json', scopes)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', scopes)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('token.json', 'w') as token:
token.write(creds.to_json())
service = build('searchconsole', 'v1', credentials=creds)
return service
First we set the variable creds
to None
. This is a common method in Python within functions, especially when you’ll be using a variable for conditionals.
Next we check to see if your folder has a token.json
file in it. If this is your first time authorizing, you don’t. If it’s not, we’re setting the creds
variable to the credentials we’ve previously gotten.
I’m going to jump around for a second because the next line we check if the credentials don’t exist or if they aren’t valid. If they exist and are valid, we completely skip this block.
If the credentials do exist but they are expired and have a refresh token, we are requesting a refresh.
If they don’t exist and don’t have a refresh token (which is the path our code will take the first time we run it), we open credentials.json
and ask for authorization. When you run this, what will happen at this point is the script will open a link in your browser and take you to the OAuth Consent Screen that you previously set up. Once you authorize that the app (script) can connect using your credentials.json
file and we set creds
to the authorization token we get back. For future use, we save the creds
into token.json
.
One important note about token.json
. You can use this same token for all the other Google API scripts we’ll be writing, if and only if the scopes don’t change. If the scopes are different, we will need to delete token.json
and re-run the function.
Finally in our function, we build the service we’ll be using for our API calls. In this case, the service is searchconsole
. And the return value for the function is service
.
Set Scopes & Obtain Authorization
Now that we have the function written, we are ready to put it to use.
The first thing we need to do is set the scopes. Since this article is only about Google Search Console, we’re only going to use the GSC scopes. You might want to include additional scopes. I normally add multiple scopes because I use the same token.json
file across several APIs. Remember, if you change your scopes, you have to delete your existing token.json
file.
You can find the available scopes in the various APIs’ documentations. Here are the GSC scopes.
scopes = ['https://www.googleapis.com/auth/webmasters']
Simple enough. There is also a .readonly version if you only ever plan on pulling data down. I prefer the non-read-only version because I might want to, for example, add or delete sitemaps some day.
Now with the scopes set, we can call our function. Set it to a variable because we’ll be using that variable for future report requests.
service = gsc_auth(scopes)
Now you can run the code!
Please visit this URL to authorize this application: https://accounts.google.com/o/oauth2/auth?THESEPARAMETERSAREGIVENTOYOU
Your browser should automatically open the link that the script creates. However, if it doesn’t, you can copy and paste the output URL. This will take you to the OAuth consent screen you set up. First, select the account you want to use (which should be the one you set the credentials up for), then you’ll see this screen.
Click Allow and you’re good to go!
You are now ready to start making requests from the Google Search Console API!
Full Code
# Import Modules
import os.path
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials
# Define function to get authorization
def gsc_auth(scopes):
creds = None
# The file token.json stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token.json'):
creds = Credentials.from_authorized_user_file('token.json', scopes)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', scopes)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('token.json', 'w') as token:
token.write(creds.to_json())
service = build('searchconsole', 'v1', credentials=creds)
return service
scopes = ['https://www.googleapis.com/auth/webmasters']
service = gsc_auth(scopes)
Eric is a Python SEO with a passion for data. He uses Python to automate tasks and analyze large data sets. He is often the go-to for spreadsheet challenges, too. He sees every challenge as an opportunity to learn.
Running this Google Collab, getting all the way to allowing access to Google but then getting an error of not localhost refusing to connect. Should I be using web application for credentials?
I haven’t run this in Google Colab myself. If I remember correctly, there’s a quicker way to authenticate the user in Google Colab because you already have to be logged into Google to use a Colab notebook. Alternatively, you could probably copy the credentials and token from a client-side authenticated run into the Colab folders so it refers to those.
I have a question about token.json. Does it expire? Do we have to refresh them every once in a while?
Yes, token.json expires but if you follow the code in here, it will refresh automatically. That’s what the
if creds and creds.expired and creds.refresh_token
statement and following code is designed to do.Thank you very much. The code refreshed the token.json. Works like a charm!
Hi Eric. Thank you very much!
I also copied the version from Sheets hehehe
My problem is that after approving and allowing the usage I get redirected to localhost (I am doing this in Colab) and I get the typical error “Cannot access this website”.
I don’t know if you are familiar with this :S
Best regards
Hi Diego. I haven’t authenticated in this way through Google Colab. I think there are other ways to authenticate through Colab since you’re already in the Google environment and logged in. But I do have another solution if you have access to create your
token.json
locally. You can addtoken.json
andcredentials.json
to your Google Drive, then mount the drive in Colab and reference the files from there. Becausetoken.json
would be pre-authorized, as long as you aren’t changing the scopes, it should work from there.