Site   Web

July 5, 2019

WHOIS Database Download: 6 Contenders to Fight Cybercriminals

SPONSORED

Image courtesy of (noomhh) / FreeDigitalPhotos.net

The Web is a dangerous place. It’s a fact. But it only becomes deadly when we cannot identify the possible sources of attacks and, consequently, are not prepared when they happen. 

You see, each year bad actors get more creative assuming fake identities and setting up networks of new domains that pop up and vanish, sometimes in a matter of hours. This means that cybersecurity professionals face a daunting task trying to catch such a shrewd and elusive enemy.

Fortunately, the caretakers of the Internet had thought it wise to do away with the anonymity by establishing the WHOIS protocol which has enabled and turned WHOIS database download services into one of the essential tools for cybersecurity specialists — allowing them to find all the identifiable information about domains. This includes their owners, registration details, phone numbers, and even information about who previously registered them and when. 

So in this article, I’ll talk about the best WHOIS database download services that can lift the masks and shine the spotlight leading to the ultimate apprehension of cybercrime perpetrators. But before we dive in, let’s briefly talk about the overall relevance of WHOIS for cybersecurity.

Table of contents

  • WHOIS database download as a pillar of cybersecurity
  • What to look for in the best WHOIS database download products
  • What’s out there: products and providers
  • Provider 1: whoisxmlapi.com/whois-database-download
  • Provider 2: domainnamestat.com/whois-database-download
  • Provider 3: iqwhois.com/whois-database-download
  • Provider 4: jsonwhois.com/whois-database-download 
  • Provider 5: whoisology.com/whois-database-download
  • Provider 6: whoisdatabasedownload.com

WHOIS Database Download as a Pillar of Cybersecurity

It’s not hard to imagine the role of WHOIS databases in maintaining cybersecurity. First of all, they can be used to verify suspicious characters that may be plotting an attack or track down cybercriminals who use all sorts of schemes to conceal their identities. 

However, an equally important function that a WHOIS database download service can perform is allowing experts to look into the domain data and the infrastructure surrounding them in order to identify threats and devise ways to stop them.

Specifically, experts can examine the registration details of entities that claim to have been in the business for a long time but whose actual records show they have actually just been registered last week or month — plausibly spreading doubts about what their intentions really are. 

Newly-registered domains are of particular interest in this regard too because they have been proven to precede hacking attacks and could appear and disappear quickly as soon as they have served their purpose. This is not to say that all newly-registered domains are getting set up for malicious ends, but as a rule of thumb, they deserve a closer look through the WHOIS prism, just to be sure.

Additionally, spotting one dangerous domain name can support unveiling a whole bunch of them at once. Indeed, hackers may operate as lone wolves or they might be part of highly-organized criminal organizations. Either way, they rarely use just one domain at a time to fool their victims and they may make the mistake of providing the same details across registrations done in bulk. See the pattern here?

What’s more, access to WHOIS databases also strengthens proactive efforts designed to prevent damaging and costly data breaches. For instance, threat hunters can cross-check data from their various sources with domain registration details and look for inconsistencies that can give away plotters. Apart from this, WHOIS data can be incorporated into threat intelligence platforms to analyze hosting configurations and help gather evidenced-based data to fortify network infrastructures.

What to Look for in the Best WHOIS Database Download Products

A WHOIS database download service can be a solid foundation able to support many cybersecurity applications, but only if it is fully-equipped to handle all the technical requirements and operational demands from a variety of potential users. It’s surely a difficult yet laudable role which can be accomplished by meeting the following criteria:

  1. Number of domains — WHOIS database download services must be able to provide accurate data on as many domain names as possible. There are now almost two billion websites worldwide, and the database that can offer all or most of them would be in the best position to serve various cybersecurity use cases. 
  1. Exhaustive data output — The best WHOIS database download service can also be determined according to the amount of domain information that it contains. The data should include important details such as the names of domain owners, email and physical addresses, contact numbers, dates of registration and expiration, and many more. Registrar information must also be available in case a user wants to report any malicious activity.
  1. TLD coverage — A WHOIS database download service should ideally cover generic top-level domains (gTLDs), country-code TLDs (ccTLDs), or new TLDs (nTLDs) to allow users to access the relevant data for whatever type of business or geographical location.
  1. Parsing quality — It is important for domain data to be appropriately structured in the database in order to be understood by users. This means that WHOIS databases should come in an easily readable format or programming language. It would be a plus if users could be given different choices of downloading formats to facilitate a smooth integration.
  1. Frequency of update — Tens of thousands of new domains are being registered on a daily basis. For this reason, a WHOIS database download must be regularly and frequently updated to provide both current and historic value. As much as possible, the information should be available as soon as a domain is registered.
  1. API access — The best WHOIS database download services should also have their own API for quick access to the data and to streamline operations. 

What’s Out There: Products and Providers

When looking for the best WHOIS database download provider, one should remember that not all products are the same. The differences may be due to a vendor wanting to concentrate more on one aspect of the service over the others for some strategic reason or purpose.

For instance, there are WHOIS database downloads that provide exclusive custom reports on, say, ccTLDs or new gTLDs. Another may do so only for gTLDs, while the rest may not think that it’s necessary to customize reports at all.

All these may be attempts to focus on functionalities that cybersecurity professionals will be looking for or find useful. But whatever the differences are, users must make an effort to evaluate each WHOIS database download service according to how it fits their specific needs.

For example, those professionals who are focused on monitoring new domains can benefit from a WHOIS database download that provides automatic notifications whenever newcomers are registered. Specialists interested in bringing down malicious infrastructure can partner with an exhaustive reverse WHOIS database provider that allows them to track down connected domains with ease.

Indeed, being clear on what cybersecurity task you want to achieve is crucial. After all, the best WHOIS database download application should be measured according to how well it will answer the questions that will be thrown its way and not just on how many features it has in store. 

Having said that, let’s review the following providers which have differentiated themselves by offering their own takes on offering a WHOIS database download service. We will rate each of them according to the criteria we have set. But again, it is up to you to ultimately decide which one would fit your idea of the most appropriate service and its capabilities.

Provider 1: whoisxmlapi.com/whois-database-download

WhoisXML API, which I am the proud founder, and its team has been compiling WHOIS records for more than ten years — accumulating a sizable database over time and satisfying the need of more than 52,000 customers. 

At the moment, WhoisXML API offers several types of database downloads. Users can avail of a classic WHOIS database download, as well as a “newly registered and just expired domains” service, and more. All their downloads are parsed and normalized to a consistent format and allow easy integration with existing business processes. They can also be customized depending on customers’ requirements. The company also offers a set of domain research and monitoring tools which can complement the WHOIS database download service. 

Here’s how the product fared according to the criteria discussed.

Number of domains — The database contains more than 1.2 billion domains and subdomains which account for 99.5% of all domains in operation. Also included are 300 million active domain names. Moreover, the dataset is growing at the rate of hundreds of thousands of domains and adjacent WHOIS records per day.

Data output  — WhoisXML API has more than 5.2 billion historic WHOIS records that include registrant name, organization, e-mail address, registration address, registrar information, creation date, expiration date, updated date, domain availability, domain age, and more. 

TLD coverage — More than 2,864 types of TLDs and ccTLDs are included in the databases.

Parsing quality It’s possible to download information in XML, JSON, MYSQL, MYSQL dump, and CSV file formats. Additionally, each record contains all parsed fields of the domain’s data allowing it to be easily processed for whatever application specialists may be using.

Frequency of update  — The database is updated daily which is especially important for cybersecurity professionals as they must keep up with current developments and pay close attention to the domain landscape.

API access — Whois XML API provides real-time APIs, including a WHOIS API ensuring quick access to the information. Additionally, the API can be used as an application for the Splunk platform so specialists can conduct WHOIS search right from within it.

Provider 2: domainnamestat.com/whois-database-download

Like their name suggests, Domain Name Stat specializes in domain statistics as the company gathers, analyzes, and processes key trends with regard to particular domain names offering their clients domain name registration statistics.

However, going through their website shows that they also provide an up-to-date historic WHOIS database download service enabling access to the past and current information on all domains that have ever been registered. The company describes its database as ‘exhaustive’ having monitored the WHOIS records of all domains since 2008.

Number of domains — This provider counts 300+ million active domain names in its database. The number represents a significant percentage of the approximately 333.8 million registered websites as of the first quarter of 2018.

Data output — Users can help themselves to 5 billion past and current WHOIS records. They include critical information on domain owners, e-mail and registration addresses, and contact numbers. 

It is also possible to find out who registered the domains, as well as the domains’ age and expiry dates, the dates when they were last updated, and many more useful data such as billing name, administrator’s name, and tech support professional details.

TLD coverage — This database supports all types of domains which cover 2,864 TLDs and ccTLDs. The former includes .com, .org, .net, .us, .biz, .info, .mobi, .coop, .pro, and .asia, while the latter include .fr, .uk, and many more.

Parsing quality — Users are given the option of receiving a duly-parsed historic WHOIS database download in different formats, either as an MYSQL, MYSQL dump, or CSV file.

Frequency of update — The database is constantly updated which means that when users purchase a complete WHOIS database download, they are provided with access to future updates including data about new domain names.

API access — All the data that are available on this provider’s website can be accessed through a real-time API which is made available through a simple pricing structure.

Provider 3: iqwhois.com/whois-database-download

IQWhois is a reverse WHOIS domain name ownership database that can be useful for tracking connections between domain names and their owners, cross-referencing details, and monitoring brands. So based on its functionality it can be considered a research tool.

Though IQWhois does not provide a live WHOIS lookup service and the product does not include all domain extensions, the company behind it, nevertheless, offers a large WHOIS database which is available for full downloadable access. Those interested in doing so have the option of having the database customized according to their specific requirements.

Number of domains — IQWhois has 300+ million active domain names, and this number, according to the company, is growing each quarter.

Data output — The database contains 5 billion historic WHOIS records which include the organization, the name of the registrant, e-mail and registration addresses, registrar information, creation and expiry dates, the dates when the domains were last updated, the domains’ ages, and many more important details. Archived data is also available on many domain names. 

Specialists conducting investigations might find interesting that the data can be used to research either individual domains and their owners or entire portfolios.

TLD coverage — The company covers 2,864+ TLDs and ccTLDs. They include .com, .net, .org, .us, .info, .pro, .biz, .mobi, .coop, .asia, .uk, .fr, .cn, .ru, and many more. 

Parsing quality — Users are given the choice to download the database in either MYSQL, MYSQL dump, or CSV file format.

Frequency of update — This provider claims to update the database regularly.

API access — Currently, the company does not offer the interface. 

Provider 4: jsonwhois.com/whois-database-download

JsonWHOIS is a domain API services provider offering historic WHOIS data for all domains. Customers can download partial or complete WHOIS database download which can be customized according to business needs.

Number of domains — At the moment they also have close to 300 million active domain names with complete historic WHOIS records.

Data output — Outputs contain complete domain information including names, addresses, phone numbers, registration dates, and many more.

TLD coverage — JsonWHOIS covers active WHOIS records for both gTLDs and ccTLDs. The 1,000+ gTLDs available include .com, .net, .org, .us, .biz, .mobi, .info, .pro, coop, .asia, and many new gTLDs. The hundreds of ccTLDs include uk, .fr, .cn, .ru, among others. 

Parsing quality — Users can get both parsed and raw WHOIS data for download as MYSQL, MYSQL dump, or CSV file formats.

Frequency of update — The database is regularly maintained and updated weekly.

API access — The company provides a WHOIS API.

Aside from the WHOIS database download service and the corresponding API, JsonWHOIS also offers a Screenshot API. Users can grab a full-page screenshot of any domain with an option to either thumbnail it or display as is. So for specialists investigating how malicious websites grow and evolve, using WHOIS API simultaneously with the Screenshot API can come in handy, making a pair of quite useful research tools.  

Provider 5: whoisology.com/whois-database-download

Whoisology claims to offer more than just data but a comprehensive and well-structured solution. That would resonate with specialists who are not just after the information per se but who are interested in the product that can be integrated into an existing system to support their cybersecurity applications.

This provider basically provides a domain name ownership archive with a database containing numerous searchable and cross-referenced domain WHOIS records. This is not a standard WHOIS lookup website but rather a database mainly focused on reverse WHOIS which can be especially useful for InfoSec investigations. As a result, users can gain access to historical WHOIS data which the company has been collecting since 2008.

Number of domains — Whoisology’s database contains 317+ million active domain names.

Data output — Like others, the service provides 5+ billion WHOIS records. Users can find out essential domain data such as ownership details, registrar information, registration and expiry dates, who to contact if there are any questions about the domain name, plus much more information depending on the specific requirements.

TLD coverage — Whoisology covers more than 2850 TLDs and ccTLDs. That includes 1,246 gTLDs (e.g., — .com, .net, .org, .biz, .info, plus more) and 1,623 ccTLDs (e.g., — .uk, .fr, .cn, .ru, and more).

Parsing quality — Users can download the database in either MYSQL, MYSQL dump, or CSV file format. The output contains all of the analyzed WHOIS domain data fields which can be processed by any application.

Frequency of update — The database is updated daily.

API access — Whoisology data is available through a dedicated API. 

Provider 6: whoisdatabasedownload.com

Whois Database Download claims to provide partial, complete, or customized historic domain WHOIS information. The service covers newly-registered domains, country-specific database and recently-expiring ones, and contains TLD domain lists and ccTLD domain lists. Upon subscription, users get 30 days of historical data since they are provided with instant access to the newly-registered domain database of the past 30 days.

The company also offers a country-specific WHOIS database that includes US, UK, Canada, Australia, India, France, Brazil, Germany, Spain, Russia, UAE, and many more countries separately. This can be quite convenient for specialists focused on investigations in a particular region or those only interested in keeping track of the domain space in a particular country where operations reside. Several options are also available for those users who want to purchase a multi-country database. 

Number of domains — The website provides users with access to more than 40 million active domain names.

Data output — Users can get up-to-date domain information including the names of domain owners, their e-mail and registration addresses, important registrar information, dates of registration and expiration, dates when domains were last updated, domain ages, and many more. Archived data is also available on many domain names.

TLD coverage — Whois Database Download has been in the business of gathering domain WHOIS records for almost all TLDs, gTLDs, and ccTLDs. They also claim to support all domain extensions.

Parsing quality — This provider’s database is available for download in CSV format. 

Frequency of update — The database is updated daily, so purchasing the complete WHOIS database download enables users to receive all future updates.

API access — This company’s WHOIS data are provided through real-time APIs which enable quick access and easy integration into a company’s system.

Another interesting aspect of this WHOIS database download service is a provision that allows customers the use, for testing purposes, of free samples of the WHOIS database for expired and registered domains.

Cybersecurity professionals have a handful of choices once it’s time to choose the best WHOIS database download. This article has taken an in-depth look at each of them — including their similarities and differences — to help in the selection, which, however, should ultimately be decided according to how well a service meets an organization’s unique specifications.

As noted earlier, I am the founder of WhoisXML API which means that I have first-hand information on the features and capabilities of the WHOIS database download product category. I welcome feedback or questions on whoisxmlapi.com or at support@whoisxmlapi.com.


avatar

Jonathan Zhang is the founder and CEO of Threat Intelligence Platform (TIP) — a data, tool, and API provider that specializes in automated threat detection, security analysis and threat intelligence solutions for Fortune 1000 and cyber-security companies. TIP is part of the Whois API Inc. family which is a trusted intelligence vendor by over 50,000 clients.

css.php