This tutorial will learn how to automate Google Search with Python. We will use multiple libraries to perform Google using Python code, making it easier for you to scrape search results. The first library we will use to perform Google searches is googlesearch-python and the second is Google.
These Python libraries will make scraping search results from Google a piece of cake. With a few lines of code, you will get the first “n” Google search results.
You can also use Selenium or BeautifulSoup for automating Google searches, but that would require more effort and understanding of how web scraping works in Python.
For now, we will use googlesearch-python and google to automate the process of getting results from Google efficiently. You can copy-paste the code and extend its data analysis and visualization functionality.
Also Read: Sending Emails Using Python With Image And PDF Attachments
Automate Google search with Python using googlesearch-python
Install the googlesearch-python library using this command: python pip install googlesearch-python
After that, you can use the Python code below to get the first ten search results from Google.
from googlesearch import search
keywordsList = ["codeing software for kids", "codeitbro", "python"]
def Results(query):
Results= search(query, num_results=10)
for i in range(len(Results)):
print(Results[i])
for i in range(len(keywordsList)):
query = keywordsList[i]
print("\n-----Search Results of----->", query)
Results(query)
We have created a list “keywordsList” to store the search terms in the above code. These search terms or keywords are passed as the Results(query) function parameters.
This Results function then uses the search() method of the library to fetch the Google results. Here is the syntax of the search() function.
search(query, num_results=x) where,
query is the search term, and num_results is to specify how many results you would like it to fetch. For example, we wanted to get only the first ten results, so we used “num_results=10” as the parameter.
Also Read: How To Make A Simple Python 3 Calculator Using Functions
You can also use another search() function variation to get search results for different languages.
Let’s say you want to get the first n search results in the French language; then you can use the lang=”fr” as an additional parameter like this:
search(query, lang="fr")
The only downside of this library is that it doesn’t allow you to change the gTLDs of Google. But don’t worry, that is where our second Python library, “google,” comes into play.
Let’s make the code to automate Google searches more comprehensive by leveraging the google Python library.
Also Read: How To Make A Digital Clock In Python Using Tkinter
Automating multiple search queries with Python using google
Install google Python library using this command: python3 pip install google
After that, you can use the code below to perform Google searches. The best part is that the code will ask users how many Google searches they want to perform and then display first n search results for their search terms.
from googlesearch import search
total_searches = int(input("How many searches you would like to do: "))
searchQueries = []
while total_searches>0:
searchTerm = input("Please enter your search term ")
for j in search(searchTerm, tld="co.in", num=10, stop=10, pause=3):
print(j)
total_searches=total_searches-1
The above code is self-explanatory, and the main thing here is the search() function, using which we are fetching the search results.
Here are the parameters of the function:
- query: keywords you would like to perform a Google search
- TLD: top-level domain of Google
- lang: the language of search results
- num: total results you want to fetch from Google
- start: first result you want to fetch
- stop: the last result you want to fetch from Google results
- pause: time lapse between the HTTP requests. A short time pause might result in Google blocking your IP address.
- Return: Iterator that yields found URLs. If the stop parameter is none, then the iterator will loop forever
Output
Also Read: How To Use Python For Browser Games Development?
Wrapping Up
This tutorial will learn how to perform Google searches using Python. Both Python libraries mentioned here will let you fetch first n search results from Google. I liked the google Python package a bit more as it lets you specify pause time and fetch search results from different TLDs of Google.
Please feel free to play with the code and extend its functionality by applying data analysis and visualization concepts. If you want to work on this project, connect with me on LinkedIn or email.
