Last Updated on:
This tutorial will teach you how to automate Google Searches with Python. We will use multiple libraries to perform Google searches using Python code, making it easier for you to scrape search results. The first library we will use to perform Google searches is Googlesearch-python, and the second is Google.
These Python libraries will make scraping search results from Google a piece of cake. With a few lines of code, you will get the first “n” Google search results.
You can also use Selenium or BeautifulSoup to automate Google searches, but that would require more effort and an understanding of how web scraping works in Python.
For now, we will use GoogleSearchPython and Google to efficiently automate getting results from Google. You can copy-paste the code and extend its data analysis and visualization functionality.
Also Read: Sending Emails Using Python With Image And PDF Attachments
Automate Google search with Python using googlesearch-python
Install the googlesearch-python library using this command: python pip install googlesearch-python
After that, you can use the Python code below to get the first ten search results from Google.
from googlesearch import search
keywordsList = ["codeing software for kids", "codeitbro", "python"]
def Results(query):
Results= search(query, num_results=10)
for i in range(len(Results)):
print(Results[i])
for i in range(len(keywordsList)):
query = keywordsList[i]
print("\n-----Search Results of----->", query)
Results(query)
We have created a list “keywordsList” to store the search terms in the above code. These search terms or keywords are passed as the Results(query) function parameters.
This Results function then fetches the Google results using the library’s search() method. Here is the syntax of the search() function.
search(query, num_results=x) where,
query is the search term, and num_results specifies how many results you would like it to fetch. For example, we wanted to get only the first ten results, so we used “num_results=10″ as the parameter.
You can also use another search function variation to get search results for different languages.
Let’s say you want to get the first n search results in the French language; then you can use the lang=”fr” as an additional parameter like this:
search(query, lang="fr")
The only downside of this library is that it doesn’t allow you to change Google’s gTLDs. But don’t worry—that is where our second Python library, Google, comes into play.
By leveraging the Google Python library, let’s make the code to automate Google searches more comprehensive.
Also Read: How To Make A Digital Clock In Python Using Tkinter
Automating multiple search queries with Python using Google
Install the Google Python library using this command: python3 pip install google
After that, you can use the code below to perform Google searches. The best part is that the code will ask users how many Google searches they want to perform and then display the first n search results for their search terms.
from googlesearch import search
total_searches = int(input("How many searches you would like to do: "))
searchQueries = []
while total_searches>0:
searchTerm = input("Please enter your search term ")
for j in search(searchTerm, tld="co.in", num=10, stop=10, pause=3):
print(j)
total_searches=total_searches-1
The above code is self-explanatory. The main thing here is the search() function, which fetches the search results.
Here are the parameters of the function:
- Query: keywords you would like to perform a Google search
- TLD: top-level domain of Google
- lang: the language of search results
- num: total results you want to fetch from Google
- start: first result you want to fetch
- stop: the last result you want to fetch from Google results
- pause: time lapse between the HTTP requests. A short pause might result in Google blocking your IP address.
- Return: Iterator that yields found URLs. If the stop parameter is none, then the iterator will loop forever
Output
Also Read: How To Use Python For Browser Games Development?
Wrapping Up
This tutorial will teach you how to perform Google searches using Python. Both Python libraries mentioned here will let you fetch the first n search results from Google. I liked the Google Python package a bit more as it enables you to specify a pause time and fetch search results from different TLDs of Google.
Please feel free to play with the code and extend its functionality by applying data analysis and visualization concepts. If you want to work on this project, connect with me on LinkedIn or email.
Other Python resources you should check:
