shodan-1.26.1/0000775000175000017500000000000014172031225014736 5ustar achilleanachillean00000000000000shodan-1.26.1/shodan.egg-info/0000775000175000017500000000000014172031225017704 5ustar achilleanachillean00000000000000shodan-1.26.1/shodan.egg-info/dependency_links.txt0000664000175000017500000000000114172031225023752 0ustar achilleanachillean00000000000000 shodan-1.26.1/shodan.egg-info/requires.txt0000664000175000017500000000013614172031225022304 0ustar achilleanachillean00000000000000click click-plugins colorama requests>=2.2.1 XlsxWriter [:python_version <= "2.7"] ipaddress shodan-1.26.1/shodan.egg-info/PKG-INFO0000664000175000017500000000751514172031225021011 0ustar achilleanachillean00000000000000Metadata-Version: 2.1 Name: shodan Version: 1.26.1 Summary: Python library and command-line utility for Shodan (https://developer.shodan.io) Home-page: https://github.com/achillean/shodan-python Author: John Matherly Author-email: jmath@shodan.io License: UNKNOWN Description: shodan: The official Python library and CLI for Shodan ====================================================== .. image:: https://img.shields.io/pypi/v/shodan.svg :target: https://pypi.org/project/shodan/ .. image:: https://img.shields.io/github/contributors/achillean/shodan-python.svg :target: https://github.com/achillean/shodan-python/graphs/contributors Shodan is a search engine for Internet-connected devices. Google lets you search for websites, Shodan lets you search for devices. This library provides developers easy access to all of the data stored in Shodan in order to automate tasks and integrate into existing tools. Features -------- - Search Shodan - `Fast/ bulk IP lookups `_ - Streaming API support for real-time consumption of Shodan firehose - `Network alerts (aka private firehose) `_ - `Manage Email Notifications `_ - Exploit search API fully implemented - Bulk data downloads - Access the Shodan DNS DB to view domain information - `Command-line interface `_ .. image:: https://cli.shodan.io/img/shodan-cli-preview.png :target: https://asciinema.org/~Shodan :width: 400px :align: center Quick Start ----------- .. code-block:: python from shodan import Shodan api = Shodan('MY API KEY') # Lookup an IP ipinfo = api.host('8.8.8.8') print(ipinfo) # Search for websites that have been "hacked" for banner in api.search_cursor('http.title:"hacked by"'): print(banner) # Get the total number of industrial control systems services on the Internet ics_services = api.count('tag:ics') print('Industrial Control Systems: {}'.format(ics_services['total'])) Grab your API key from https://account.shodan.io Installation ------------ To install the Shodan library, simply: .. code-block:: bash $ pip install shodan Or if you don't have pip installed (which you should seriously install): .. code-block:: bash $ easy_install shodan Documentation ------------- Documentation is available at https://shodan.readthedocs.org/ and https://help.shodan.io Keywords: security,network Platform: UNKNOWN Classifier: Development Status :: 5 - Production/Stable Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: MIT License Classifier: Natural Language :: English Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 2.6 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.4 Classifier: Programming Language :: Python :: 3.5 Classifier: Programming Language :: Python :: 3.6 Classifier: Topic :: Software Development :: Libraries :: Python Modules Description-Content-Type: text/x-rst shodan-1.26.1/shodan.egg-info/entry_points.txt0000644000175000017500000000006114172031225023175 0ustar achilleanachillean00000000000000[console_scripts] shodan = shodan.__main__:main shodan-1.26.1/shodan.egg-info/SOURCES.txt0000664000175000017500000000203714172031225021572 0ustar achilleanachillean00000000000000.gitignore AUTHORS CHANGELOG.md LICENSE MANIFEST.in README.rst requirements.txt setup.py docs/Makefile docs/api.rst docs/conf.py docs/index.rst docs/make.bat docs/tutorial.rst docs/examples/basic-search.rst docs/examples/cert-stream.rst docs/examples/gifcreator.rst docs/examples/query-summary.rst shodan/__init__.py shodan/__main__.py shodan/client.py shodan/exception.py shodan/helpers.py shodan/stream.py shodan/threatnet.py shodan.egg-info/PKG-INFO shodan.egg-info/SOURCES.txt shodan.egg-info/dependency_links.txt shodan.egg-info/entry_points.txt shodan.egg-info/requires.txt shodan.egg-info/top_level.txt shodan/cli/__init__.py shodan/cli/alert.py shodan/cli/data.py shodan/cli/helpers.py shodan/cli/host.py shodan/cli/organization.py shodan/cli/scan.py shodan/cli/settings.py shodan/cli/worldmap.py shodan/cli/converter/__init__.py shodan/cli/converter/base.py shodan/cli/converter/csvc.py shodan/cli/converter/excel.py shodan/cli/converter/geojson.py shodan/cli/converter/images.py shodan/cli/converter/kml.py tests/__init__.py tests/test_shodan.pyshodan-1.26.1/shodan.egg-info/top_level.txt0000664000175000017500000000000714172031225022433 0ustar achilleanachillean00000000000000shodan shodan-1.26.1/README.rst0000664000175000017500000000432113725547165016447 0ustar achilleanachillean00000000000000shodan: The official Python library and CLI for Shodan ====================================================== .. image:: https://img.shields.io/pypi/v/shodan.svg :target: https://pypi.org/project/shodan/ .. image:: https://img.shields.io/github/contributors/achillean/shodan-python.svg :target: https://github.com/achillean/shodan-python/graphs/contributors Shodan is a search engine for Internet-connected devices. Google lets you search for websites, Shodan lets you search for devices. This library provides developers easy access to all of the data stored in Shodan in order to automate tasks and integrate into existing tools. Features -------- - Search Shodan - `Fast/ bulk IP lookups `_ - Streaming API support for real-time consumption of Shodan firehose - `Network alerts (aka private firehose) `_ - `Manage Email Notifications `_ - Exploit search API fully implemented - Bulk data downloads - Access the Shodan DNS DB to view domain information - `Command-line interface `_ .. image:: https://cli.shodan.io/img/shodan-cli-preview.png :target: https://asciinema.org/~Shodan :width: 400px :align: center Quick Start ----------- .. code-block:: python from shodan import Shodan api = Shodan('MY API KEY') # Lookup an IP ipinfo = api.host('8.8.8.8') print(ipinfo) # Search for websites that have been "hacked" for banner in api.search_cursor('http.title:"hacked by"'): print(banner) # Get the total number of industrial control systems services on the Internet ics_services = api.count('tag:ics') print('Industrial Control Systems: {}'.format(ics_services['total'])) Grab your API key from https://account.shodan.io Installation ------------ To install the Shodan library, simply: .. code-block:: bash $ pip install shodan Or if you don't have pip installed (which you should seriously install): .. code-block:: bash $ easy_install shodan Documentation ------------- Documentation is available at https://shodan.readthedocs.org/ and https://help.shodan.io shodan-1.26.1/tests/0000775000175000017500000000000014172031225016100 5ustar achilleanachillean00000000000000shodan-1.26.1/tests/test_shodan.py0000664000175000017500000001130213430134602020761 0ustar achilleanachillean00000000000000import unittest import shodan try: basestring except NameError: basestring = str class ShodanTests(unittest.TestCase): api = None FACETS = [ 'port', ('domain', 1) ] QUERIES = { 'simple': 'cisco-ios', 'minify': 'apache', 'advanced': 'apache port:443', 'empty': 'asdasdasdasdasdasdasdasdasdhjihjkjk', } def setUp(self): self.api = shodan.Shodan(open('SHODAN-API-KEY').read().strip()) def test_search_simple(self): results = self.api.search(self.QUERIES['simple']) # Make sure the properties exist self.assertIn('matches', results) self.assertIn('total', results) # Make sure no error occurred self.assertNotIn('error', results) # Make sure some values were returned self.assertTrue(results['matches']) self.assertTrue(results['total']) # A regular search shouldn't have the optional info self.assertNotIn('opts', results['matches'][0]) def test_search_empty(self): results = self.api.search(self.QUERIES['empty']) self.assertTrue(len(results['matches']) == 0) self.assertEqual(results['total'], 0) def test_search_facets(self): results = self.api.search(self.QUERIES['simple'], facets=self.FACETS) self.assertTrue(results['facets']['port']) self.assertEqual(len(results['facets']['domain']), 1) def test_count_simple(self): results = self.api.count(self.QUERIES['simple']) # Make sure the properties exist self.assertIn('matches', results) self.assertIn('total', results) # Make sure no error occurred self.assertNotIn('error', results) # Make sure no values were returned self.assertFalse(results['matches']) self.assertTrue(results['total']) def test_count_facets(self): results = self.api.count(self.QUERIES['simple'], facets=self.FACETS) self.assertTrue(results['facets']['port']) self.assertEqual(len(results['facets']['domain']), 1) def test_host_details(self): host = self.api.host('147.228.101.7') self.assertEqual('147.228.101.7', host['ip_str']) self.assertFalse(isinstance(host['ip'], basestring)) def test_search_minify(self): results = self.api.search(self.QUERIES['minify'], minify=False) self.assertIn('opts', results['matches'][0]) def test_exploits_search(self): results = self.api.exploits.search('apache') self.assertIn('matches', results) self.assertIn('total', results) self.assertTrue(results['matches']) def test_exploits_search_paging(self): results = self.api.exploits.search('apache', page=1) match1 = results['matches'][0] results = self.api.exploits.search('apache', page=2) match2 = results['matches'][0] self.assertNotEqual(match1['_id'], match2['_id']) def test_exploits_search_facets(self): results = self.api.exploits.search('apache', facets=['source', ('author', 1)]) self.assertIn('facets', results) self.assertTrue(results['facets']['source']) self.assertTrue(len(results['facets']['author']) == 1) def test_exploits_count(self): results = self.api.exploits.count('apache') self.assertIn('matches', results) self.assertIn('total', results) self.assertTrue(len(results['matches']) == 0) def test_exploits_count_facets(self): results = self.api.exploits.count('apache', facets=['source', ('author', 1)]) self.assertEqual(len(results['matches']), 0) self.assertIn('facets', results) self.assertTrue(results['facets']['source']) self.assertTrue(len(results['facets']['author']) == 1) # Test error responses def test_invalid_key(self): api = shodan.Shodan('garbage') raised = False try: api.search('something') except shodan.APIError: raised = True self.assertTrue(raised) def test_invalid_host_ip(self): raised = False try: self.api.host('test') except shodan.APIError: raised = True self.assertTrue(raised) def test_search_empty_query(self): raised = False try: self.api.search('') except shodan.APIError: raised = True self.assertTrue(raised) def test_search_advanced_query(self): # The free API plan can't use filters raised = False try: self.api.search(self.QUERIES['advanced']) except shodan.APIError: raised = True self.assertTrue(raised) if __name__ == '__main__': unittest.main() shodan-1.26.1/tests/__init__.py0000664000175000017500000000000012414103244020175 0ustar achilleanachillean00000000000000shodan-1.26.1/CHANGELOG.md0000664000175000017500000001166314003643320016554 0ustar achilleanachillean00000000000000CHANGELOG ========= 1.25.0 ------ * Add new CLI command: shodan alert download 1.24.0 ------ * Add new CLI command: shodan alert stats 1.23.0 ------ * Add new CLI command: shodan alert domain 1.22.1 ------ * Fix bug when converting data file to CSV using Python3 1.22.0 ------ * Add support for new vulnerability streaming endpoints 1.21.3 ------ * Fix geo.json file converter 1.21.2 ------ * Add support for paging through the domain information 1.21.1 ------ * Add ``history`` and ``type`` parameters to ``Shodan.dns.domain_info()`` method and CLI command 1.21.0 ------ * New API methods ``api.search_facets()`` and ``api.search_filters()`` to get a list of available facets and filters. 1.20.0 ------ * New option "-S" for **shodan domain** to save results from the lookup * New option "-D" for **shodan domain** to lookup open ports for IPs in the results 1.19.0 ------ * New method to edit the list of IPs for an existing network alert 1.18.0 ------ * Add library methods for the new Notifications API 1.17.0 ------ * Fix bug that caused unicode error when printing domain information (#106) * Add flag to let users get their IPv6 address **shodan myip -6**(#35) 1.16.0 ------ * Ability to specify list of fields to include when converting to CSV/ Excel (#107) * Filter the Shodan Firehose based on tags in the banner 1.15.0 ------ * New option "--skip" for download command to help users resume a download 1.14.0 ------ * New command **shodan version** (#104). * Only change api_key file permissions if needed (#103) 1.13.0 ------ * New command **shodan domain** to lookup a domain in Shodan's DNS database * Override environment configured settings if explicit proxy settings are supplied (@cudeso) 1.12.1 ------ * Fix Excel file conversion that resulted in empty .xlsx files 1.12.0 ------ * Add new methods to ignore/ unignore trigger notifications 1.11.1 ------ * Allow a single network alert to monitor multiple IP ranges (#93) 1.11.0 ------ * New command **shodan scan list** to list recently launched scans * New command **shodan alert triggers** to list the available notification triggers * New command **shodan alert enable** to enable a notification trigger * New command **shodan alert disable** to disable a notification trigger * New command **shodan alert info** to show details of a specific alert * Include timestamp, vulns and tags in CSV converter (#85) * Fixed bug that caused an exception when parsing uncompressed data files in Python3 * Code quality improvements * Thank you for contributions from @wagner-certat, @cclauss, @opt9, @voldmar and Antoine Neuenschwander 1.10.4 ------ * Fix a bug when showing old banner records that don't have the "transport" property * Code quality improvements (bare excepts) 1.10.3 ------ * Change bare 'except:' statements to 'except Exception:' or more specific ones * remove unused imports * Convert line endings of `shodan/client.py` and `tests/test_shodan.py` to unix * List file types in **shodan convert** (#80) 1.10.2 ------ * Fix **shodan stats** formatting exception when faceting on **port** 1.10.1 ------ * Support PUT requests in the API request helper method 1.10.0 ------ * New command **shodan org**: manage enterprise access to Shodan for your team * Improved unicode handling (#78) * Remove deprecated API wrapper for shodanhq.com/api 1.9.1 ----- * The CHANGELOG is now part of the packages. * Improved unicode handling in Python2 (#78) * Add `tsv` output format for **shodan host** (#65) * Show user-friendly error messages when running **shodan radar** without permission or in a window that's too small (#74) * Improved exception handling to improve debugging **shodan init** (#77) 1.9.0 ----- * New optional parameter `proxies` for all interfaces to specify a proxy array for the requests library (#72) 1.8.1 ----- * Fixed bug that prevented **shodan scan submit** from finishing (#70) 1.8.0 ----- * Shodan CLI now installs properly on Windows (#66) * Improved output of "shodan host" (#64, #67) * Fixed bug that prevented an open port from being shown in "shodan host" (#63) * No longer show an empty page if "shodan search" didn't return results (#62) * Updated docs to make them Python3 compatible 1.7.7 ----- * Added "shodan data download" command to help download bulk data files 1.7.6 ----- * Add basic support for the Bulk Data API 1.7.5 ----- * Handle Cloudflare timeouts 1.7.4 ----- * Added "shodan radar" command 1.7.3 ----- * Fixed the bug #47 which was caused by the CLI using a timeout value of "0" which resulted in the "requests" library failing to connect 1.7.2 ----- * stream: automatically decode to unicode, fixes streaming on python3 (#45) * Include docs in packages (#46) * stream: handle timeout=None, None (default) can't be compared with integers (#44) 1.7.1 ----- * Python3 fixes for outputting images (#42) * Add the ability to save results from host lookups via the CLI (#43) 1.7.0 ----- * Added "images" convert output format to let users extract images from Shodan data files (#42) shodan-1.26.1/PKG-INFO0000664000175000017500000000751514172031225016043 0ustar achilleanachillean00000000000000Metadata-Version: 2.1 Name: shodan Version: 1.26.1 Summary: Python library and command-line utility for Shodan (https://developer.shodan.io) Home-page: https://github.com/achillean/shodan-python Author: John Matherly Author-email: jmath@shodan.io License: UNKNOWN Description: shodan: The official Python library and CLI for Shodan ====================================================== .. image:: https://img.shields.io/pypi/v/shodan.svg :target: https://pypi.org/project/shodan/ .. image:: https://img.shields.io/github/contributors/achillean/shodan-python.svg :target: https://github.com/achillean/shodan-python/graphs/contributors Shodan is a search engine for Internet-connected devices. Google lets you search for websites, Shodan lets you search for devices. This library provides developers easy access to all of the data stored in Shodan in order to automate tasks and integrate into existing tools. Features -------- - Search Shodan - `Fast/ bulk IP lookups `_ - Streaming API support for real-time consumption of Shodan firehose - `Network alerts (aka private firehose) `_ - `Manage Email Notifications `_ - Exploit search API fully implemented - Bulk data downloads - Access the Shodan DNS DB to view domain information - `Command-line interface `_ .. image:: https://cli.shodan.io/img/shodan-cli-preview.png :target: https://asciinema.org/~Shodan :width: 400px :align: center Quick Start ----------- .. code-block:: python from shodan import Shodan api = Shodan('MY API KEY') # Lookup an IP ipinfo = api.host('8.8.8.8') print(ipinfo) # Search for websites that have been "hacked" for banner in api.search_cursor('http.title:"hacked by"'): print(banner) # Get the total number of industrial control systems services on the Internet ics_services = api.count('tag:ics') print('Industrial Control Systems: {}'.format(ics_services['total'])) Grab your API key from https://account.shodan.io Installation ------------ To install the Shodan library, simply: .. code-block:: bash $ pip install shodan Or if you don't have pip installed (which you should seriously install): .. code-block:: bash $ easy_install shodan Documentation ------------- Documentation is available at https://shodan.readthedocs.org/ and https://help.shodan.io Keywords: security,network Platform: UNKNOWN Classifier: Development Status :: 5 - Production/Stable Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: MIT License Classifier: Natural Language :: English Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 2.6 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.4 Classifier: Programming Language :: Python :: 3.5 Classifier: Programming Language :: Python :: 3.6 Classifier: Topic :: Software Development :: Libraries :: Python Modules Description-Content-Type: text/x-rst shodan-1.26.1/AUTHORS0000664000175000017500000000006412414103244016004 0ustar achilleanachillean00000000000000Primary authors: * John Matherly shodan-1.26.1/.gitignore0000664000175000017500000000015613416545200016733 0ustar achilleanachillean00000000000000build/* *.tar.gz *.json.gz *.kml *.egg *.pyc shodan.egg-info/* tmp/* MANIFEST .vscode/ PKG-INFO venv/* .idea/*shodan-1.26.1/setup.cfg0000664000175000017500000000004614172031225016557 0ustar achilleanachillean00000000000000[egg_info] tag_build = tag_date = 0 shodan-1.26.1/requirements.txt0000664000175000017500000000012714003371304020220 0ustar achilleanachillean00000000000000click click-plugins colorama requests>=2.2.1 XlsxWriter ipaddress;python_version<='2.7'shodan-1.26.1/setup.py0000755000175000017500000000261714172031170016456 0ustar achilleanachillean00000000000000#!/usr/bin/env python from setuptools import setup DEPENDENCIES = open('requirements.txt', 'r').read().split('\n') README = open('README.rst', 'r').read() setup( name='shodan', version='1.26.1', description='Python library and command-line utility for Shodan (https://developer.shodan.io)', long_description=README, long_description_content_type='text/x-rst', author='John Matherly', author_email='jmath@shodan.io', url='https://github.com/achillean/shodan-python', packages=['shodan', 'shodan.cli', 'shodan.cli.converter'], entry_points={'console_scripts': ['shodan=shodan.__main__:main']}, install_requires=DEPENDENCIES, keywords=['security', 'network'], classifiers=[ 'Development Status :: 5 - Production/Stable', 'Intended Audience :: Developers', 'License :: OSI Approved :: MIT License', 'Natural Language :: English', 'Operating System :: OS Independent', 'Programming Language :: Python', 'Programming Language :: Python :: 2', 'Programming Language :: Python :: 2.6', 'Programming Language :: Python :: 2.7', 'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3.4', 'Programming Language :: Python :: 3.5', 'Programming Language :: Python :: 3.6', 'Topic :: Software Development :: Libraries :: Python Modules', ], ) shodan-1.26.1/shodan/0000775000175000017500000000000014172031225016212 5ustar achilleanachillean00000000000000shodan-1.26.1/shodan/cli/0000775000175000017500000000000014172031225016761 5ustar achilleanachillean00000000000000shodan-1.26.1/shodan/cli/alert.py0000664000175000017500000003574314003643157020464 0ustar achilleanachillean00000000000000import click import csv import shodan from collections import defaultdict from operator import itemgetter from shodan import APIError from shodan.cli.helpers import get_api_key from shodan.helpers import open_file, write_banner from time import sleep MAX_QUERY_LENGTH = 1000 def aggregate_facet(api, networks, facets): """Merge the results from multiple facet API queries into a single result object. This is necessary because a user might be monitoring a lot of IPs/ networks so it doesn't fit into a single API call. """ def _merge_custom_facets(lfacets, results): for key in results['facets']: if key not in lfacets: lfacets[key] = defaultdict(int) for item in results['facets'][key]: lfacets[key][item['value']] += item['count'] # We're going to create a custom facets dict where # the key is the value of a facet. Normally the facets # object is a list where each item has a "value" and "count" property. tmp_facets = {} count = 0 query = 'net:' for net in networks: query += '{},'.format(net) # Start running API queries if the query length is getting long if len(query) > MAX_QUERY_LENGTH: results = api.count(query[:-1], facets=facets) _merge_custom_facets(tmp_facets, results) count += results['total'] query = 'net:' # Run any remaining search query if query[-1] != ':': results = api.count(query[:-1], facets=facets) _merge_custom_facets(tmp_facets, results) count += results['total'] # Convert the internal facets structure back to the one that # the API returns. new_facets = {} for facet in tmp_facets: sorted_items = sorted(tmp_facets[facet].items(), key=itemgetter(1), reverse=True) new_facets[facet] = [{'value': key, 'count': value} for key, value in sorted_items] # Make sure the facet keys exist even if there weren't any results for facet, _ in facets: if facet not in new_facets: new_facets[facet] = [] return { 'matches': [], 'facets': new_facets, 'total': count, } @click.group() def alert(): """Manage the network alerts for your account""" pass @alert.command(name='clear') def alert_clear(): """Remove all alerts""" key = get_api_key() # Get the list api = shodan.Shodan(key) try: alerts = api.alerts() for alert in alerts: click.echo(u'Removing {} ({})'.format(alert['name'], alert['id'])) api.delete_alert(alert['id']) except shodan.APIError as e: raise click.ClickException(e.value) click.echo("Alerts deleted") @alert.command(name='create') @click.argument('name', metavar='') @click.argument('netblocks', metavar='', nargs=-1) def alert_create(name, netblocks): """Create a network alert to monitor an external network""" key = get_api_key() # Get the list api = shodan.Shodan(key) try: alert = api.create_alert(name, netblocks) except shodan.APIError as e: raise click.ClickException(e.value) click.secho('Successfully created network alert!', fg='green') click.secho('Alert ID: {}'.format(alert['id']), fg='cyan') @alert.command(name='domain') @click.argument('domain', metavar='', type=str) @click.option('--triggers', help='List of triggers to enable', default='malware,industrial_control_system,internet_scanner,iot,open_database,new_service,ssl_expired,vulnerable') def alert_domain(domain, triggers): """Create a network alert based on a domain name""" key = get_api_key() api = shodan.Shodan(key) try: # Grab a list of IPs for the domain domain = domain.lower() click.secho('Looking up domain information...', dim=True) info = api.dns.domain_info(domain, type='A') domain_ips = set([record['value'] for record in info['data']]) # Create the actual alert click.secho('Creating alert...', dim=True) alert = api.create_alert('__domain: {}'.format(domain), list(domain_ips)) # Enable the triggers so it starts getting managed by Shodan Monitor click.secho('Enabling triggers...', dim=True) api.enable_alert_trigger(alert['id'], triggers) except shodan.APIError as e: raise click.ClickException(e.value) click.secho('Successfully created domain alert!', fg='green') click.secho('Alert ID: {}'.format(alert['id']), fg='cyan') @alert.command(name='download') @click.argument('filename', metavar='', type=str) @click.option('--alert-id', help='Specific alert ID to download the data of', default=None) def alert_download(filename, alert_id): """Download all information for monitored networks/ IPs.""" key = get_api_key() api = shodan.Shodan(key) ips = set() networks = set() # Helper method to process batches of IPs def batch(iterable, size=1): iter_length = len(iterable) for ndx in range(0, iter_length, size): yield iterable[ndx:min(ndx + size, iter_length)] try: # Get the list of alerts for the user click.echo('Looking up alert information...') if alert_id: alerts = [api.alerts(aid=alert_id.strip())] else: alerts = api.alerts() click.echo('Compiling list of networks/ IPs to download...') for alert in alerts: for net in alert['filters']['ip']: if '/' in net: networks.add(net) else: ips.add(net) click.echo('Downloading...') with open_file(filename) as fout: # Check if the user is able to use batch IP lookups batch_size = 1 if len(ips) > 0: api_info = api.info() if api_info['plan'] in ['corp', 'stream-100']: batch_size = 100 # Convert it to a list so we can index into it ips = list(ips) # Grab all the IP information for ip in batch(ips, size=batch_size): try: click.echo(ip) results = api.host(ip) if not isinstance(results, list): results = [results] for host in results: for banner in host['data']: write_banner(fout, banner) except APIError: pass sleep(1) # Slow down a bit to make sure we don't hit the rate limit # Grab all the network ranges for net in networks: try: counter = 0 click.echo(net) for banner in api.search_cursor('net:{}'.format(net)): write_banner(fout, banner) # Slow down a bit to make sure we don't hit the rate limit if counter % 100 == 0: sleep(1) counter += 1 except APIError: pass except shodan.APIError as e: raise click.ClickException(e.value) click.secho('Successfully downloaded results into: {}'.format(filename), fg='green') @alert.command(name='info') @click.argument('alert', metavar='') def alert_info(alert): """Show information about a specific alert""" key = get_api_key() api = shodan.Shodan(key) try: info = api.alerts(aid=alert) except shodan.APIError as e: raise click.ClickException(e.value) click.secho(info['name'], fg='cyan') click.secho('Created: ', nl=False, dim=True) click.secho(info['created'], fg='magenta') click.secho('Notifications: ', nl=False, dim=True) if 'triggers' in info and info['triggers']: click.secho('enabled', fg='green') else: click.echo('disabled') click.echo('') click.secho('Network Range(s):', dim=True) for network in info['filters']['ip']: click.echo(u' > {}'.format(click.style(network, fg='yellow'))) click.echo('') if 'triggers' in info and info['triggers']: click.secho('Triggers:', dim=True) for trigger in info['triggers']: click.echo(u' > {}'.format(click.style(trigger, fg='yellow'))) click.echo('') @alert.command(name='list') @click.option('--expired', help='Whether or not to show expired alerts.', default=True, type=bool) def alert_list(expired): """List all the active alerts""" key = get_api_key() # Get the list api = shodan.Shodan(key) try: results = api.alerts(include_expired=expired) except shodan.APIError as e: raise click.ClickException(e.value) if len(results) > 0: click.echo(u'# {:14} {:<21} {:<15s}'.format('Alert ID', 'Name', 'IP/ Network')) for alert in results: click.echo( u'{:16} {:<30} {:<35} '.format( click.style(alert['id'], fg='yellow'), click.style(alert['name'], fg='cyan'), click.style(', '.join(alert['filters']['ip']), fg='white') ), nl=False ) if 'triggers' in alert and alert['triggers']: click.secho('Triggers: ', fg='magenta', nl=False) click.echo(', '.join(alert['triggers'].keys()), nl=False) if 'expired' in alert and alert['expired']: click.secho('expired', fg='red') else: click.echo('') else: click.echo("You haven't created any alerts yet.") @alert.command(name='stats') @click.option('--limit', help='The number of results to return.', default=10, type=int) @click.option('--filename', '-O', help='Save the results in a CSV file of the provided name.', default=None) @click.argument('facets', metavar='', nargs=-1) def alert_stats(limit, filename, facets): """Show summary information about your monitored networks""" # Setup Shodan key = get_api_key() api = shodan.Shodan(key) # Make sure the user didn't supply an empty string if not facets: raise click.ClickException('No facets provided') facets = [(facet, limit) for facet in facets] # Get the list of IPs/ networks that the user is monitoring networks = set() try: alerts = api.alerts() for alert in alerts: for tmp in alert['filters']['ip']: networks.add(tmp) except shodan.APIError as e: raise click.ClickException(e.value) # Grab the facets the user requested try: results = aggregate_facet(api, networks, facets) except shodan.APIError as e: raise click.ClickException(e.value) # TODO: The below code was taken from __main__.py:stats() - we should refactor it so the code can be shared # Print the stats tables for facet in results['facets']: click.echo('Top {} Results for Facet: {}'.format(len(results['facets'][facet]), facet)) for item in results['facets'][facet]: # Force the value to be a string - necessary because some facet values are numbers value = u'{}'.format(item['value']) click.echo(click.style(u'{:28s}'.format(value), fg='cyan'), nl=False) click.echo(click.style(u'{:12,d}'.format(item['count']), fg='green')) click.echo('') # Create the output file if requested fout = None if filename: if not filename.endswith('.csv'): filename += '.csv' fout = open(filename, 'w') writer = csv.writer(fout, dialect=csv.excel) # Write the header that contains the facets row = [] for facet in results['facets']: row.append(facet) row.append('') writer.writerow(row) # Every facet has 2 columns (key, value) counter = 0 has_items = True while has_items: # pylint: disable=W0612 row = ['' for i in range(len(results['facets']) * 2)] pos = 0 has_items = False for facet in results['facets']: values = results['facets'][facet] # Add the values for the facet into the current row if len(values) > counter: has_items = True row[pos] = values[counter]['value'] row[pos + 1] = values[counter]['count'] pos += 2 # Write out the row if has_items: writer.writerow(row) # Move to the next row of values counter += 1 @alert.command(name='remove') @click.argument('alert_id', metavar='') def alert_remove(alert_id): """Remove the specified alert""" key = get_api_key() # Get the list api = shodan.Shodan(key) try: api.delete_alert(alert_id) except shodan.APIError as e: raise click.ClickException(e.value) click.echo("Alert deleted") @alert.command(name='triggers') def alert_list_triggers(): """List the available notification triggers""" key = get_api_key() # Get the list api = shodan.Shodan(key) try: results = api.alert_triggers() except shodan.APIError as e: raise click.ClickException(e.value) if len(results) > 0: click.secho('The following triggers can be enabled on alerts:', dim=True) click.echo('') for trigger in sorted(results, key=itemgetter('name')): click.secho('{:<12} '.format('Name'), dim=True, nl=False) click.secho(trigger['name'], fg='yellow') click.secho('{:<12} '.format('Description'), dim=True, nl=False) click.secho(trigger['description'], fg='cyan') click.secho('{:<12} '.format('Rule'), dim=True, nl=False) click.echo(trigger['rule']) click.echo('') else: click.echo("No triggers currently available.") @alert.command(name='enable') @click.argument('alert_id', metavar='') @click.argument('trigger', metavar='') def alert_enable_trigger(alert_id, trigger): """Enable a trigger for the alert""" key = get_api_key() # Get the list api = shodan.Shodan(key) try: api.enable_alert_trigger(alert_id, trigger) except shodan.APIError as e: raise click.ClickException(e.value) click.secho('Successfully enabled the trigger: {}'.format(trigger), fg='green') @alert.command(name='disable') @click.argument('alert_id', metavar='') @click.argument('trigger', metavar='') def alert_disable_trigger(alert_id, trigger): """Disable a trigger for the alert""" key = get_api_key() # Get the list api = shodan.Shodan(key) try: api.disable_alert_trigger(alert_id, trigger) except shodan.APIError as e: raise click.ClickException(e.value) click.secho('Successfully disabled the trigger: {}'.format(trigger), fg='green') shodan-1.26.1/shodan/cli/worldmap.py0000775000175000017500000002217613674472212021206 0ustar achilleanachillean00000000000000''' F-Secure Virus World Map console edition See README.md for more details Copyright 2012-2013 Jyrki Muukkonen Released under the MIT license. See LICENSE.txt or http://www.opensource.org/licenses/mit-license.php ASCII map in map-world-01.txt is copyright: "Map 1998 Matthew Thomas. Freely usable as long as this line is included" ''' import curses import locale import random import time from shodan.exception import APIError from shodan.helpers import get_ip MAPS = { 'world': { # offset (as (y, x) for curses...) 'corners': (1, 4, 23, 73), # lat top, lon left, lat bottom, lon right 'coords': [90.0, -180.0, -90.0, 180.0], # PyLint freaks out about the world map backslashes so ignore those warnings 'data': r''' . _..::__: ,-"-"._ |7 , _,.__ _.___ _ _<_>`!(._`.`-. / _._ `_ ,_/ ' '-._.---.-.__ .{ " " `-==,',._\{ \ / {) / _ ">_,-' ` mt-2_ \_.:--. `._ )`^-. "' , [_/( __,/-' '"' \ " _L oD_,--' ) /. (| | ,' _)_.\\._<> 6 _,' / ' `. / [_/_'` `"( <'} ) \\ .-. ) / `-'"..' `:._ _) ' ` \ ( `( / `:\ > \ ,-^. /' ' `._, "" | \`' \| ?_) {\ `=.---. `._._ ,' "` |' ,- '. | `-._ | / `:`<_|h--._ ( > . | , `=.__.`-'\ `. / | |{| ,-.,\ . | ,' \ / `' ," \ | / |_' | __ / | | '-' `-' \. |/ " / \. ' ,/ ______._.--._ _..---.---------._ ,-----"-..?----_/ ) _,-'" " ( Map 1998 Matthew Thomas. Freely usable as long as this line is included ''' } } class AsciiMap(object): """ Helper class for handling map drawing and coordinate calculations """ def __init__(self, map_name='world', map_conf=None, window=None, encoding=None): if map_conf is None: map_conf = MAPS[map_name] self.map = map_conf['data'] self.coords = map_conf['coords'] self.corners = map_conf['corners'] if window is None: window = curses.newwin(0, 0) self.window = window self.data = [] self.data_timestamp = None # JSON contents _should_ be UTF8 (so, python internal unicode here...) if encoding is None: encoding = locale.getpreferredencoding() self.encoding = encoding # check if we can use transparent background or not if curses.can_change_color(): curses.use_default_colors() background = -1 else: background = curses.COLOR_BLACK tmp_colors = [ ('red', curses.COLOR_RED, background), ('blue', curses.COLOR_BLUE, background), ('pink', curses.COLOR_MAGENTA, background) ] self.colors = {} if curses.has_colors(): for i, (name, fgcolor, bgcolor) in enumerate(tmp_colors, 1): curses.init_pair(i, fgcolor, bgcolor) self.colors[name] = i def latlon_to_coords(self, lat, lon): """ Convert lat/lon coordinates to character positions. Very naive version, assumes that we are drawing the whole world TODO: filter out stuff that doesn't fit TODO: make it possible to use "zoomed" maps """ width = (self.corners[3] - self.corners[1]) height = (self.corners[2] - self.corners[0]) # change to 0-180, 0-360 abs_lat = -lat + 90 abs_lon = lon + 180 x = (abs_lon / 360.0) * width + self.corners[1] y = (abs_lat / 180.0) * height + self.corners[0] return int(x), int(y) def set_data(self, data): """ Set / convert internal data. For now it just selects a random set to show. """ entries = [] # Grab 5 random banners to display for banner in random.sample(data, min(len(data), 5)): desc = '{} -> {} / {}'.format(get_ip(banner), banner['port'], banner['location']['country_code']) if banner['location']['city']: # Not all cities can be encoded in ASCII so ignore any errors try: desc += ' {}'.format(banner['location']['city']) except Exception: pass if 'tags' in banner and banner['tags']: desc += ' / {}'.format(','.join(banner['tags'])) entry = ( float(banner['location']['latitude']), float(banner['location']['longitude']), '*', desc, curses.A_BOLD, 'red', ) entries.append(entry) self.data = entries def draw(self, target): """ Draw internal data to curses window """ self.window.clear() self.window.addstr(0, 0, self.map) # FIXME: position to be defined in map config? row = self.corners[2] - 6 items_to_show = 5 for lat, lon, char, desc, attrs, color in self.data: # to make this work almost everywhere. see http://docs.python.org/2/library/curses.html if desc: desc = desc.encode(self.encoding, 'ignore').decode() if items_to_show <= 0: break char_x, char_y = self.latlon_to_coords(lat, lon) if self.colors and color: attrs |= curses.color_pair(self.colors[color]) self.window.addstr(char_y, char_x, char, attrs) if desc: det_show = "%s %s" % (char, desc) else: det_show = None if det_show is not None: try: self.window.addstr(row, 1, det_show, attrs) row += 1 items_to_show -= 1 except Exception: # FIXME: check window size before addstr() break self.window.overwrite(target) self.window.leaveok(1) class MapApp(object): """ Virus World Map ncurses application """ def __init__(self, api): self.api = api self.data = None self.last_fetch = 0 self.sleep = 10 # tenths of seconds, for curses.halfdelay() self.polling_interval = 60 def fetch_data(self, epoch_now, force_refresh=False): """ (Re)fetch data from JSON stream """ refresh = False if force_refresh or self.data is None: refresh = True else: if self.last_fetch + self.polling_interval <= epoch_now: refresh = True if refresh: try: # Grab 20 banners from the main stream banners = [] for banner in self.api.stream.banners(): if 'location' in banner and banner['location']['latitude']: banners.append(banner) if len(banners) >= 20: break self.data = banners self.last_fetch = epoch_now except APIError: raise return refresh def run(self, scr): """ Initialize and run the application """ m = AsciiMap() curses.halfdelay(self.sleep) while True: now = int(time.time()) refresh = self.fetch_data(now) m.set_data(self.data) try: m.draw(scr) except curses.error: raise Exception('Terminal window too small') scr.addstr(0, 1, 'Shodan Radar', curses.A_BOLD) scr.addstr(0, 40, time.strftime("%c UTC", time.gmtime(now)).rjust(37), curses.A_BOLD) # Key Input # q - Quit event = scr.getch() if event == ord('q'): break # redraw window (to fix encoding/rendering bugs and to hide other messages to same tty) # user pressed 'r' or new data was fetched if refresh: m.window.redrawwin() def launch_map(api): app = MapApp(api) return curses.wrapper(app.run) def main(argv=None): """ Main function / entry point """ from shodan import Shodan from shodan.cli.helpers import get_api_key api = Shodan(get_api_key()) return launch_map(api) if __name__ == '__main__': import sys sys.exit(main()) shodan-1.26.1/shodan/cli/organization.py0000664000175000017500000000426213430142637022051 0ustar achilleanachillean00000000000000import click import shodan from shodan.cli.helpers import get_api_key, humanize_api_plan @click.group() def org(): """Manage your organization's access to Shodan""" pass @org.command() @click.option('--silent', help="Don't send a notification to the user", default=False, is_flag=True) @click.argument('user', metavar='') def add(silent, user): """Add a new member""" key = get_api_key() api = shodan.Shodan(key) try: api.org.add_member(user, notify=not silent) except shodan.APIError as e: raise click.ClickException(e.value) click.secho('Successfully added the new member', fg='green') @org.command() def info(): """Show an overview of the organization""" key = get_api_key() api = shodan.Shodan(key) try: organization = api.org.info() except shodan.APIError as e: raise click.ClickException(e.value) click.secho(organization['name'], fg='cyan') click.secho('Access Level: ', nl=False, dim=True) click.secho(humanize_api_plan(organization['upgrade_type']), fg='magenta') if organization['domains']: click.secho('Authorized Domains: ', nl=False, dim=True) click.echo(', '.join(organization['domains'])) click.echo('') click.secho('Administrators:', dim=True) for admin in organization['admins']: click.echo(u' > {:30}\t{:30}'.format( click.style(admin['username'], fg='yellow'), admin['email']) ) click.echo('') if organization['members']: click.secho('Members:', dim=True) for member in organization['members']: click.echo(u' > {:30}\t{:30}'.format( click.style(member['username'], fg='yellow'), member['email']) ) else: click.secho('No members yet', dim=True) @org.command() @click.argument('user', metavar='') def remove(user): """Remove and downgrade a member""" key = get_api_key() api = shodan.Shodan(key) try: api.org.remove_member(user) except shodan.APIError as e: raise click.ClickException(e.value) click.secho('Successfully removed the member', fg='green') shodan-1.26.1/shodan/cli/data.py0000664000175000017500000000635213563034516020263 0ustar achilleanachillean00000000000000import click import requests import shodan import shodan.helpers as helpers from shodan.cli.helpers import get_api_key @click.group() def data(): """Bulk data access to Shodan""" pass @data.command(name='list') @click.option('--dataset', help='See the available files in the given dataset', default=None, type=str) def data_list(dataset): """List available datasets or the files within those datasets.""" # Setup the API connection key = get_api_key() api = shodan.Shodan(key) if dataset: # Show the files within this dataset files = api.data.list_files(dataset) for file in files: click.echo(click.style(u'{:20s}'.format(file['name']), fg='cyan'), nl=False) click.echo(click.style('{:10s}'.format(helpers.humanize_bytes(file['size'])), fg='yellow'), nl=False) # Show the SHA1 checksum if available if file.get('sha1'): click.echo(click.style('{:42s}'.format(file['sha1']), fg='green'), nl=False) click.echo('{}'.format(file['url'])) else: # If no dataset was provided then show a list of all datasets datasets = api.data.list_datasets() for ds in datasets: click.echo(click.style('{:15s}'.format(ds['name']), fg='cyan'), nl=False) click.echo('{}'.format(ds['description'])) @data.command(name='download') @click.option('--chunksize', help='The size of the chunks that are downloaded into memory before writing them to disk.', default=1024, type=int) @click.option('--filename', '-O', help='Save the file as the provided filename instead of the default.') @click.argument('dataset', metavar='') @click.argument('name', metavar='') def data_download(chunksize, filename, dataset, name): # Setup the API connection key = get_api_key() api = shodan.Shodan(key) # Get the file object that the user requested which will contain the URL and total file size file = None try: files = api.data.list_files(dataset) for tmp in files: if tmp['name'] == name: file = tmp break except shodan.APIError as e: raise click.ClickException(e.value) # The file isn't available if not file: raise click.ClickException('File not found') # Start downloading the file response = requests.get(file['url'], stream=True) # Figure out the size of the file based on the headers filesize = response.headers.get('content-length', None) if not filesize: # Fall back to using the filesize provided by the API filesize = file['size'] else: filesize = int(filesize) chunk_size = 1024 limit = filesize / chunk_size # Create a default filename based on the dataset and the filename within that dataset if not filename: filename = '{}-{}'.format(dataset, name) # Open the output file and start writing to it in chunks with open(filename, 'wb') as fout: with click.progressbar(response.iter_content(chunk_size=chunk_size), length=limit) as bar: for chunk in bar: if chunk: fout.write(chunk) click.echo(click.style('Download completed: {}'.format(filename), 'green')) shodan-1.26.1/shodan/cli/__init__.py0000664000175000017500000000000012436430126021064 0ustar achilleanachillean00000000000000shodan-1.26.1/shodan/cli/helpers.py0000664000175000017500000000721613563035251021011 0ustar achilleanachillean00000000000000''' Helper methods used across the CLI commands. ''' import click import datetime import gzip import itertools import os import sys from ipaddress import ip_network, ip_address from .settings import SHODAN_CONFIG_DIR try: basestring # Python 2 except NameError: basestring = (str, ) # Python 3 def get_api_key(): '''Returns the API key of the current logged-in user.''' shodan_dir = os.path.expanduser(SHODAN_CONFIG_DIR) keyfile = shodan_dir + '/api_key' # If the file doesn't yet exist let the user know that they need to # initialize the shodan cli if not os.path.exists(keyfile): raise click.ClickException('Please run "shodan init " before using this command') # Make sure it is a read-only file if not oct(os.stat(keyfile).st_mode).endswith("600"): os.chmod(keyfile, 0o600) with open(keyfile, 'r') as fin: return fin.read().strip() def escape_data(args): # Make sure the string is unicode so the terminal can properly display it # We do it using format() so it works across Python 2 and 3 args = u'{}'.format(args) return args.replace('\n', '\\n').replace('\r', '\\r').replace('\t', '\\t') def timestr(): return datetime.datetime.utcnow().strftime('%Y-%m-%d') def open_streaming_file(directory, timestr, compresslevel=9): return gzip.open('%s/%s.json.gz' % (directory, timestr), 'a', compresslevel) def get_banner_field(banner, flat_field): # The provided field is a collapsed form of the actual field fields = flat_field.split('.') try: current_obj = banner for field in fields: current_obj = current_obj[field] return current_obj except Exception: pass return None def filter_with_netmask(banner, netmask): # filtering based on netmask is a more abstract concept than # a mere check for a specific field and thus needs its own mechanism # this will enable users to use the net:10.0.0.0/8 syntax they are used to # to find specific networks from a big shodan download. network = ip_network(netmask) ip_field = get_banner_field(banner, 'ip') if not ip_field: return False banner_ip_address = ip_address(ip_field) return banner_ip_address in network def match_filters(banner, filters): for args in filters: flat_field, check = args.split(':', 1) if flat_field == 'net': return filter_with_netmask(banner, check) value = get_banner_field(banner, flat_field) # If the field doesn't exist on the banner then ignore the record if not value: return False # It must match all filters to be allowed field_type = type(value) # For lists of strings we see whether the desired value is contained in the field if field_type == list or isinstance(value, basestring): if check not in value: return False elif field_type == int: if int(check) != value: return False elif field_type == float: if float(check) != value: return False else: # Ignore unknown types pass return True def async_spinner(finished): spinner = itertools.cycle(['-', '/', '|', '\\']) while not finished.is_set(): sys.stdout.write('\b{}'.format(next(spinner))) sys.stdout.flush() finished.wait(0.2) def humanize_api_plan(plan): return { 'oss': 'Free', 'dev': 'Membership', 'basic': 'Freelancer API', 'plus': 'Small Business API', 'corp': 'Corporate API', 'stream-100': 'Enterprise', }[plan] shodan-1.26.1/shodan/cli/converter/0000775000175000017500000000000014172031225020770 5ustar achilleanachillean00000000000000shodan-1.26.1/shodan/cli/converter/csvc.py0000664000175000017500000000533213725550715022320 0ustar achilleanachillean00000000000000 from .base import Converter from ...helpers import iterate_files try: # python 3.x: Import ABC from collections.abc from collections.abc import MutableMapping except ImportError: # Python 2.x: Import ABC from collections from collections import MutableMapping from csv import writer as csv_writer, excel class CsvConverter(Converter): fields = [ 'data', 'hostnames', 'ip', 'ip_str', 'ipv6', 'org', 'isp', 'location.country_code', 'location.city', 'location.country_name', 'location.latitude', 'location.longitude', 'os', 'asn', 'port', 'tags', 'timestamp', 'transport', 'product', 'version', 'vulns', 'ssl.cipher.version', 'ssl.cipher.bits', 'ssl.cipher.name', 'ssl.alpn', 'ssl.versions', 'ssl.cert.serial', 'ssl.cert.fingerprint.sha1', 'ssl.cert.fingerprint.sha256', 'html', 'title', ] def process(self, files): writer = csv_writer(self.fout, dialect=excel, lineterminator='\n') # Write the header writer.writerow(self.fields) for banner in iterate_files(files): # The "vulns" property can't be nicely flattened as-is so we turn # it into a list before processing the banner. if 'vulns' in banner: banner['vulns'] = list(banner['vulns'].keys()) # Python3 returns dict_keys so we neeed to cover that to a list try: row = [] for field in self.fields: value = self.banner_field(banner, field) row.append(value) writer.writerow(row) except Exception: pass def banner_field(self, banner, flat_field): # The provided field is a collapsed form of the actual field fields = flat_field.split('.') try: current_obj = banner for field in fields: current_obj = current_obj[field] # Convert a list into a concatenated string if isinstance(current_obj, list): current_obj = ','.join([str(i) for i in current_obj]) return current_obj except Exception: pass return '' def flatten(self, d, parent_key='', sep='.'): items = [] for k, v in d.items(): new_key = parent_key + sep + k if parent_key else k if isinstance(v, MutableMapping): items.extend(self.flatten(v, new_key, sep=sep).items()) else: items.append((new_key, v)) return dict(items) shodan-1.26.1/shodan/cli/converter/__init__.py0000664000175000017500000000025113430135512023077 0ustar achilleanachillean00000000000000from .csvc import CsvConverter from .excel import ExcelConverter from .geojson import GeoJsonConverter from .images import ImagesConverter from .kml import KmlConverter shodan-1.26.1/shodan/cli/converter/excel.py0000664000175000017500000000724413453722514022462 0ustar achilleanachillean00000000000000 from .base import Converter from ...helpers import iterate_files, get_ip from collections import defaultdict from xlsxwriter import Workbook class ExcelConverter(Converter): fields = [ 'port', 'timestamp', 'data', 'hostnames', 'org', 'isp', 'location.country_name', 'location.country_code', 'location.city', 'os', 'asn', 'transport', 'product', 'version', 'http.server', 'http.title', ] field_names = { 'org': 'Organization', 'isp': 'ISP', 'location.country_code': 'Country ISO Code', 'location.country_name': 'Country', 'location.city': 'City', 'os': 'OS', 'asn': 'ASN', 'http.server': 'Web Server', 'http.title': 'Website Title', } def process(self, files): # Get the filename from the already-open file handle filename = self.fout.name # Close the existing file as the XlsxWriter library handles that for us self.fout.close() # Create the new workbook workbook = Workbook(filename) # Define some common styles/ formats bold = workbook.add_format({ 'bold': 1, }) # Create the main worksheet where all the raw data is shown main_sheet = workbook.add_worksheet('Raw Data') # Write the header main_sheet.write(0, 0, 'IP', bold) # The IP field can be either ip_str or ipv6 so we treat it differently main_sheet.set_column(0, 0, 20) row = 0 col = 1 for field in self.fields: name = self.field_names.get(field, field.capitalize()) main_sheet.write(row, col, name, bold) col += 1 row += 1 total = 0 ports = defaultdict(int) for banner in iterate_files(files): try: # Build the list that contains all the relevant values data = [] for field in self.fields: value = self.banner_field(banner, field) data.append(value) # Write those values to the main workbook # Starting off w/ the special "IP" property main_sheet.write_string(row, 0, get_ip(banner)) col = 1 for value in data: main_sheet.write(row, col, value) col += 1 row += 1 except Exception: pass # Aggregate summary information total += 1 ports[banner['port']] += 1 summary_sheet = workbook.add_worksheet('Summary') summary_sheet.write(0, 0, 'Total', bold) summary_sheet.write(0, 1, total) # Ports Distribution summary_sheet.write(0, 3, 'Ports Distribution', bold) row = 1 col = 3 for key, value in sorted(ports.items(), reverse=True, key=lambda kv: (kv[1], kv[0])): summary_sheet.write(row, col, key) summary_sheet.write(row, col + 1, value) row += 1 workbook.close() def banner_field(self, banner, flat_field): # The provided field is a collapsed form of the actual field fields = flat_field.split('.') try: current_obj = banner for field in fields: current_obj = current_obj[field] # Convert a list into a concatenated string if isinstance(current_obj, list): current_obj = ','.join([str(i) for i in current_obj]) return current_obj except Exception: pass return '' shodan-1.26.1/shodan/cli/converter/base.py0000664000175000017500000000016513430155171022261 0ustar achilleanachillean00000000000000 class Converter: def __init__(self, fout): self.fout = fout def process(self, fout): pass shodan-1.26.1/shodan/cli/converter/images.py0000664000175000017500000000343513430155212022613 0ustar achilleanachillean00000000000000 from .base import Converter from ...helpers import iterate_files, get_ip, get_screenshot # Needed for decoding base64-strings in Python3 from codecs import decode import os class ImagesConverter(Converter): # The Images converter is special in that it creates a directory and there's # special code in the Shodan CLI that relies on the "dirname" property to let # the user know where the images have been stored. dirname = None def process(self, files): # Get the filename from the already-open file handle and use it as # the directory name to store the images. self.dirname = self.fout.name[:-7] + '-images' # Remove the original file that was created self.fout.close() os.unlink(self.fout.name) # Create the directory if it doesn't yet exist if not os.path.exists(self.dirname): os.mkdir(self.dirname) # Close the existing file as the XlsxWriter library handles that for us self.fout.close() # Loop through all the banners in the data file for banner in iterate_files(files): screenshot = get_screenshot(banner) if screenshot: filename = '{}/{}-{}'.format(self.dirname, get_ip(banner), banner['port']) # If a file with the name already exists then count up until we # create a new, unique filename counter = 0 tmpname = filename while os.path.exists(tmpname + '.jpg'): tmpname = '{}-{}'.format(filename, counter) counter += 1 filename = tmpname + '.jpg' fout = open(filename, 'wb') fout.write(decode(screenshot['data'].encode(), 'base64')) fout.close() shodan-1.26.1/shodan/cli/converter/kml.py0000664000175000017500000000765213430155632022144 0ustar achilleanachillean00000000000000 from .base import Converter from ...helpers import iterate_files class KmlConverter(Converter): def header(self): self.fout.write(""" """) def footer(self): self.fout.write("""""") def process(self, files): # Write the header self.header() hosts = {} for banner in iterate_files(files): ip = banner.get('ip_str', banner.get('ipv6', None)) if not ip: continue if ip not in hosts: hosts[ip] = banner hosts[ip]['ports'] = [] hosts[ip]['ports'].append(banner['port']) for ip, host in iter(hosts.items()): self.write(host) self.footer() def write(self, host): try: ip = host.get('ip_str', host.get('ipv6', None)) lat, lon = host['location']['latitude'], host['location']['longitude'] placemark = '{}]]>'.format(ip) placemark += '{0}'.format(host['hostnames'][0]) placemark += '

Ports

    ' for port in host['ports']: placemark += """
  • {}
    • """.format(port) placemark += '
' placemark += """
powered by Shodan
""".format(ip) placemark += ']]>' placemark += '{},{}'.format(lon, lat) placemark += '' self.fout.write(placemark.encode('utf-8')) except Exception: pass shodan-1.26.1/shodan/cli/converter/geojson.py0000664000175000017500000000253713642705667023040 0ustar achilleanachillean00000000000000from json import dumps from .base import Converter from ...helpers import get_ip, iterate_files class GeoJsonConverter(Converter): def header(self): self.fout.write("""{ "type": "FeatureCollection", "features": [ """) def footer(self): self.fout.write("""{ }]}""") def process(self, files): # Write the header self.header() # We only want to generate 1 datapoint for each IP - not per service unique_hosts = set() for banner in iterate_files(files): ip = get_ip(banner) if not ip: continue if ip not in unique_hosts: self.write(ip, banner) unique_hosts.add(ip) self.footer() def write(self, ip, host): try: lat, lon = host['location']['latitude'], host['location']['longitude'] feature = { 'type': 'Feature', 'id': ip, 'properties': { 'name': ip, 'lat': lat, 'lon': lon, }, 'geometry': { 'type': 'Point', 'coordinates': [lon, lat], }, } self.fout.write(dumps(feature) + ',') except Exception: pass shodan-1.26.1/shodan/cli/settings.py0000664000175000017500000000046713740703034021206 0ustar achilleanachillean00000000000000 from os import path if path.exists(path.expanduser("~/.shodan")): SHODAN_CONFIG_DIR = '~/.shodan/' else: SHODAN_CONFIG_DIR = "~/.config/shodan/" COLORIZE_FIELDS = { 'ip_str': 'green', 'port': 'yellow', 'data': 'white', 'hostnames': 'magenta', 'org': 'cyan', 'vulns': 'red', } shodan-1.26.1/shodan/cli/host.py0000664000175000017500000001164013430142745020320 0ustar achilleanachillean00000000000000# Helper methods for printing `host` information to the terminal. import click from shodan.helpers import get_ip def host_print_pretty(host, history=False): """Show the host information in a user-friendly way and try to include as much relevant information as possible.""" # General info click.echo(click.style(get_ip(host), fg='green')) if len(host['hostnames']) > 0: click.echo(u'{:25s}{}'.format('Hostnames:', ';'.join(host['hostnames']))) if 'city' in host and host['city']: click.echo(u'{:25s}{}'.format('City:', host['city'])) if 'country_name' in host and host['country_name']: click.echo(u'{:25s}{}'.format('Country:', host['country_name'])) if 'os' in host and host['os']: click.echo(u'{:25s}{}'.format('Operating System:', host['os'])) if 'org' in host and host['org']: click.echo(u'{:25s}{}'.format('Organization:', host['org'])) if 'last_update' in host and host['last_update']: click.echo('{:25s}{}'.format('Updated:', host['last_update'])) click.echo('{:25s}{}'.format('Number of open ports:', len(host['ports']))) # Output the vulnerabilities the host has if 'vulns' in host and len(host['vulns']) > 0: vulns = [] for vuln in host['vulns']: if vuln.startswith('!'): continue if vuln.upper() == 'CVE-2014-0160': vulns.append(click.style('Heartbleed', fg='red')) else: vulns.append(click.style(vuln, fg='red')) if len(vulns) > 0: click.echo('{:25s}'.format('Vulnerabilities:'), nl=False) for vuln in vulns: click.echo(vuln + '\t', nl=False) click.echo('') click.echo('') # If the user doesn't have access to SSL/ Telnet results then we need # to pad the host['data'] property with empty banners so they still see # the port listed as open. (#63) if len(host['ports']) != len(host['data']): # Find the ports the user can't see the data for ports = host['ports'] for banner in host['data']: if banner['port'] in ports: ports.remove(banner['port']) # Add the placeholder banners for port in ports: banner = { 'port': port, 'transport': 'tcp', # All the filtered services use TCP 'timestamp': host['data'][-1]['timestamp'], # Use the timestamp of the oldest banner 'placeholder': True, # Don't store this banner when the file is saved } host['data'].append(banner) click.echo('Ports:') for banner in sorted(host['data'], key=lambda k: k['port']): product = '' version = '' if 'product' in banner and banner['product']: product = banner['product'] if 'version' in banner and banner['version']: version = '({})'.format(banner['version']) click.echo(click.style('{:>7d}'.format(banner['port']), fg='cyan'), nl=False) if 'transport' in banner: click.echo('/', nl=False) click.echo(click.style('{} '.format(banner['transport']), fg='yellow'), nl=False) click.echo('{} {}'.format(product, version), nl=False) if history: # Format the timestamp to only show the year-month-day date = banner['timestamp'][:10] click.echo(click.style('\t\t({})'.format(date), fg='white', dim=True), nl=False) click.echo('') # Show optional ssl info if 'ssl' in banner: if 'versions' in banner['ssl'] and banner['ssl']['versions']: click.echo('\t|-- SSL Versions: {}'.format(', '.join([item for item in sorted(banner['ssl']['versions']) if not version.startswith('-')]))) if 'dhparams' in banner['ssl'] and banner['ssl']['dhparams']: click.echo('\t|-- Diffie-Hellman Parameters:') click.echo('\t\t{:15s}{}\n\t\t{:15s}{}'.format('Bits:', banner['ssl']['dhparams']['bits'], 'Generator:', banner['ssl']['dhparams']['generator'])) if 'fingerprint' in banner['ssl']['dhparams']: click.echo('\t\t{:15s}{}'.format('Fingerprint:', banner['ssl']['dhparams']['fingerprint'])) def host_print_tsv(host, history=False): """Show the host information in a succinct, grep-friendly manner.""" for banner in sorted(host['data'], key=lambda k: k['port']): click.echo(click.style('{:>7d}'.format(banner['port']), fg='cyan'), nl=False) click.echo('\t', nl=False) click.echo(click.style('{} '.format(banner['transport']), fg='yellow'), nl=False) if history: # Format the timestamp to only show the year-month-day date = banner['timestamp'][:10] click.echo(click.style('\t({})'.format(date), fg='white', dim=True), nl=False) click.echo('') HOST_PRINT = { 'pretty': host_print_pretty, 'tsv': host_print_tsv, } shodan-1.26.1/shodan/cli/scan.py0000664000175000017500000003405513575267152020306 0ustar achilleanachillean00000000000000import click import collections import datetime import shodan import shodan.helpers as helpers import socket import threading import time from shodan.cli.helpers import get_api_key, async_spinner from shodan.cli.settings import COLORIZE_FIELDS @click.group() def scan(): """Scan an IP/ netblock using Shodan.""" pass @scan.command(name='list') def scan_list(): """Show recently launched scans""" key = get_api_key() # Get the list api = shodan.Shodan(key) try: scans = api.scans() except shodan.APIError as e: raise click.ClickException(e.value) if len(scans) > 0: click.echo(u'# {} Scans Total - Showing 10 most recent scans:'.format(scans['total'])) click.echo(u'# {:20} {:<15} {:<10} {:<15s}'.format('Scan ID', 'Status', 'Size', 'Timestamp')) # click.echo('#' * 65) for scan in scans['matches'][:10]: click.echo( u'{:31} {:<24} {:<10} {:<15s}'.format( click.style(scan['id'], fg='yellow'), click.style(scan['status'], fg='cyan'), scan['size'], scan['created'] ) ) else: click.echo("You haven't yet launched any scans.") @scan.command(name='internet') @click.option('--quiet', help='Disable the printing of information to the screen.', default=False, is_flag=True) @click.argument('port', type=int) @click.argument('protocol', type=str) def scan_internet(quiet, port, protocol): """Scan the Internet for a specific port and protocol using the Shodan infrastructure.""" key = get_api_key() api = shodan.Shodan(key) try: # Submit the request to Shodan click.echo('Submitting Internet scan to Shodan...', nl=False) scan = api.scan_internet(port, protocol) click.echo('Done') # If the requested port is part of the regular Shodan crawling, then # we don't know when the scan is done so lets return immediately and # let the user decide when to stop waiting for further results. official_ports = api.ports() if port in official_ports: click.echo('The requested port is already indexed by Shodan. A new scan for the port has been launched, please subscribe to the real-time stream for results.') else: # Create the output file filename = '{0}-{1}.json.gz'.format(port, protocol) counter = 0 with helpers.open_file(filename, 'w') as fout: click.echo('Saving results to file: {0}'.format(filename)) # Start listening for results done = False # Keep listening for results until the scan is done click.echo('Waiting for data, please stand by...') while not done: try: for banner in api.stream.ports([port], timeout=90): counter += 1 helpers.write_banner(fout, banner) if not quiet: click.echo('{0:<40} {1:<20} {2}'.format( click.style(helpers.get_ip(banner), fg=COLORIZE_FIELDS['ip_str']), click.style(str(banner['port']), fg=COLORIZE_FIELDS['port']), ';'.join(banner['hostnames'])) ) except shodan.APIError: # We stop waiting for results if the scan has been processed by the crawlers and # there haven't been new results in a while if done: break scan = api.scan_status(scan['id']) if scan['status'] == 'DONE': done = True except socket.timeout: # We stop waiting for results if the scan has been processed by the crawlers and # there haven't been new results in a while if done: break scan = api.scan_status(scan['id']) if scan['status'] == 'DONE': done = True except Exception as e: raise click.ClickException(repr(e)) click.echo('Scan finished: {0} devices found'.format(counter)) except shodan.APIError as e: raise click.ClickException(e.value) @scan.command(name='protocols') def scan_protocols(): """List the protocols that you can scan with using Shodan.""" key = get_api_key() api = shodan.Shodan(key) try: protocols = api.protocols() for name, description in iter(protocols.items()): click.echo(click.style('{0:<30}'.format(name), fg='cyan') + description) except shodan.APIError as e: raise click.ClickException(e.value) @scan.command(name='submit') @click.option('--wait', help='How long to wait for results to come back. If this is set to "0" or below return immediately.', default=20, type=int) @click.option('--filename', help='Save the results in the given file.', default='', type=str) @click.option('--force', default=False, is_flag=True) @click.option('--verbose', default=False, is_flag=True) @click.argument('netblocks', metavar='', nargs=-1) def scan_submit(wait, filename, force, verbose, netblocks): """Scan an IP/ netblock using Shodan.""" key = get_api_key() api = shodan.Shodan(key) alert = None # Submit the IPs for scanning try: # Submit the scan scan = api.scan(netblocks, force=force) now = datetime.datetime.now().strftime('%Y-%m-%d %H:%M') click.echo('') click.echo('Starting Shodan scan at {} - {} scan credits left'.format(now, scan['credits_left'])) if verbose: click.echo('# Scan ID: {}'.format(scan['id'])) # Return immediately if wait <= 0: click.echo('Exiting now, not waiting for results. Use the API or website to retrieve the results of the scan.') else: # Setup an alert to wait for responses alert = api.create_alert('Scan: {}'.format(', '.join(netblocks)), netblocks) # Create the output file if necessary filename = filename.strip() fout = None if filename != '': # Add the appropriate extension if it's not there atm if not filename.endswith('.json.gz'): filename += '.json.gz' fout = helpers.open_file(filename, 'w') # Start a spinner finished_event = threading.Event() progress_bar_thread = threading.Thread(target=async_spinner, args=(finished_event,)) progress_bar_thread.start() # Now wait a few seconds for items to get returned hosts = collections.defaultdict(dict) done = False scan_start = time.time() cache = {} while not done: try: for banner in api.stream.alert(aid=alert['id'], timeout=wait): ip = banner.get('ip', banner.get('ipv6', None)) if not ip: continue # Don't show duplicate banners cache_key = '{}:{}'.format(ip, banner['port']) if cache_key not in cache: hosts[helpers.get_ip(banner)][banner['port']] = banner cache[cache_key] = True # If we've grabbed data for more than 60 seconds it might just be a busy network and we should move on if time.time() - scan_start >= 60: scan = api.scan_status(scan['id']) if verbose: click.echo('# Scan status: {}'.format(scan['status'])) if scan['status'] == 'DONE': done = True break except shodan.APIError: # If the connection timed out before the timeout, that means the streaming server # that the user tried to reach is down. In that case, lets wait briefly and try # to connect again! if (time.time() - scan_start) < wait: time.sleep(0.5) continue # Exit if the scan was flagged as done somehow if done: break scan = api.scan_status(scan['id']) if scan['status'] == 'DONE': done = True if verbose: click.echo('# Scan status: {}'.format(scan['status'])) except socket.timeout: # If the connection timed out before the timeout, that means the streaming server # that the user tried to reach is down. In that case, lets wait a second and try # to connect again! if (time.time() - scan_start) < wait: continue done = True except Exception as e: finished_event.set() progress_bar_thread.join() raise click.ClickException(repr(e)) finished_event.set() progress_bar_thread.join() def print_field(name, value): click.echo(' {:25s}{}'.format(name, value)) def print_banner(banner): click.echo(' {:20s}'.format(click.style(str(banner['port']), fg='green') + '/' + banner['transport']), nl=False) if 'product' in banner: click.echo(banner['product'], nl=False) if 'version' in banner: click.echo(' ({})'.format(banner['version']), nl=False) click.echo('') # Show optional ssl info if 'ssl' in banner: if 'versions' in banner['ssl']: # Only print SSL versions if they were successfully tested versions = [version for version in sorted(banner['ssl']['versions']) if not version.startswith('-')] if len(versions) > 0: click.echo(' |-- SSL Versions: {}'.format(', '.join(versions))) if 'dhparams' in banner['ssl'] and banner['ssl']['dhparams']: click.echo(' |-- Diffie-Hellman Parameters:') click.echo(' {:15s}{}\n {:15s}{}'.format('Bits:', banner['ssl']['dhparams']['bits'], 'Generator:', banner['ssl']['dhparams']['generator'])) if 'fingerprint' in banner['ssl']['dhparams']: click.echo(' {:15s}{}'.format('Fingerprint:', banner['ssl']['dhparams']['fingerprint'])) if hosts: # Remove the remaining spinner character click.echo('\b ') for ip in sorted(hosts): host = next(iter(hosts[ip].items()))[1] click.echo(click.style(ip, fg='cyan'), nl=False) if 'hostnames' in host and host['hostnames']: click.echo(' ({})'.format(', '.join(host['hostnames'])), nl=False) click.echo('') if 'location' in host and 'country_name' in host['location'] and host['location']['country_name']: print_field('Country', host['location']['country_name']) if 'city' in host['location'] and host['location']['city']: print_field('City', host['location']['city']) if 'org' in host and host['org']: print_field('Organization', host['org']) if 'os' in host and host['os']: print_field('Operating System', host['os']) click.echo('') # Output the vulnerabilities the host has if 'vulns' in host and len(host['vulns']) > 0: vulns = [] for vuln in host['vulns']: if vuln.startswith('!'): continue if vuln.upper() == 'CVE-2014-0160': vulns.append(click.style('Heartbleed', fg='red')) else: vulns.append(click.style(vuln, fg='red')) if len(vulns) > 0: click.echo(' {:25s}'.format('Vulnerabilities:'), nl=False) for vuln in vulns: click.echo(vuln + '\t', nl=False) click.echo('') # Print all the open ports: click.echo(' Open Ports:') for port in sorted(hosts[ip]): print_banner(hosts[ip][port]) # Save the banner in a file if necessary if fout: helpers.write_banner(fout, hosts[ip][port]) click.echo('') else: # Prepend a \b to remove the spinner click.echo('\bNo open ports found or the host has been recently crawled and cant get scanned again so soon.') except shodan.APIError as e: raise click.ClickException(e.value) finally: # Remove any alert if alert: api.delete_alert(alert['id']) @scan.command(name='status') @click.argument('scan_id', type=str) def scan_status(scan_id): """Check the status of an on-demand scan.""" key = get_api_key() api = shodan.Shodan(key) try: scan = api.scan_status(scan_id) click.echo(scan['status']) except shodan.APIError as e: raise click.ClickException(e.value) shodan-1.26.1/shodan/client.py0000644000175000017500000006263414165663573020076 0ustar achilleanachillean00000000000000# -*- coding: utf-8 -*- """ shodan.client ~~~~~~~~~~~~~ This module implements the Shodan API. :copyright: (c) 2014- by John Matherly """ import time import requests import json from .exception import APIError from .helpers import api_request, create_facet_string from .stream import Stream # Try to disable the SSL warnings in urllib3 since not everybody can install # C extensions. If you're able to install C extensions you can try to run: # # pip install requests[security] # # Which will download libraries that offer more full-featured SSL classes # pylint: disable=E1101 try: requests.packages.urllib3.disable_warnings() except Exception: pass # Define a basestring type if necessary for Python3 compatibility try: basestring except NameError: basestring = str class Shodan: """Wrapper around the Shodan REST and Streaming APIs :param key: The Shodan API key that can be obtained from your account page (https://account.shodan.io) :type key: str :ivar exploits: An instance of `shodan.Shodan.Exploits` that provides access to the Exploits REST API. :ivar stream: An instance of `shodan.Shodan.Stream` that provides access to the Streaming API. """ class Data: def __init__(self, parent): self.parent = parent def list_datasets(self): """Returns a list of datasets that the user has permission to download. :returns: A list of objects where every object describes a dataset """ return self.parent._request('/shodan/data', {}) def list_files(self, dataset): """Returns a list of files that belong to the given dataset. :returns: A list of objects where each object contains a 'name', 'size', 'timestamp' and 'url' """ return self.parent._request('/shodan/data/{}'.format(dataset), {}) class Dns: def __init__(self, parent): self.parent = parent def domain_info(self, domain, history=False, type=None, page=1): """Grab the DNS information for a domain. """ args = { 'page': page, } if history: args['history'] = history if type: args['type'] = type return self.parent._request('/dns/domain/{}'.format(domain), args) class Notifier: def __init__(self, parent): self.parent = parent def create(self, provider, args, description=None): """Get the settings for the specified notifier that a user has configured. :param provider: Provider name :type provider: str :param args: Provider arguments :type args: dict :param description: Human-friendly description of the notifier :type description: str :returns: dict -- fields are 'success' and 'id' of the notifier """ args['provider'] = provider if description: args['description'] = description return self.parent._request('/notifier', args, method='post') def edit(self, nid, args): """Get the settings for the specified notifier that a user has configured. :param nid: Notifier ID :type nid: str :param args: Provider arguments :type args: dict :returns: dict -- fields are 'success' and 'id' of the notifier """ return self.parent._request('/notifier/{}'.format(nid), args, method='put') def get(self, nid): """Get the settings for the specified notifier that a user has configured. :param nid: Notifier ID :type nid: str :returns: dict -- object describing the notifier settings """ return self.parent._request('/notifier/{}'.format(nid), {}) def list_notifiers(self): """Returns a list of notifiers that the user has added. :returns: A list of notifierse that are available on the account """ return self.parent._request('/notifier', {}) def list_providers(self): """Returns a list of supported notification providers. :returns: A list of providers where each object describes a provider """ return self.parent._request('/notifier/provider', {}) def remove(self, nid): """Delete the provided notifier. :param nid: Notifier ID :type nid: str :returns: dict -- 'success' set to True if action succeeded """ return self.parent._request('/notifier/{}'.format(nid), {}, method='delete') class Tools: def __init__(self, parent): self.parent = parent def myip(self): """Get your current IP address as seen from the Internet. :returns: str -- your IP address """ return self.parent._request('/tools/myip', {}) class Exploits: def __init__(self, parent): self.parent = parent def search(self, query, page=1, facets=None): """Search the entire Shodan Exploits archive using the same query syntax as the website. :param query: The exploit search query; same syntax as website. :type query: str :param facets: A list of strings or tuples to get summary information on. :type facets: str :param page: The page number to access. :type page: int :returns: dict -- a dictionary containing the results of the search. """ query_args = { 'query': query, 'page': page, } if facets: query_args['facets'] = create_facet_string(facets) return self.parent._request('/api/search', query_args, service='exploits') def count(self, query, facets=None): """Search the entire Shodan Exploits archive but only return the total # of results, not the actual exploits. :param query: The exploit search query; same syntax as website. :type query: str :param facets: A list of strings or tuples to get summary information on. :type facets: str :returns: dict -- a dictionary containing the results of the search. """ query_args = { 'query': query, } if facets: query_args['facets'] = create_facet_string(facets) return self.parent._request('/api/count', query_args, service='exploits') class Labs: def __init__(self, parent): self.parent = parent def honeyscore(self, ip): """Calculate the probability of an IP being an ICS honeypot. :param ip: IP address of the device :type ip: str :returns: int -- honeyscore ranging from 0.0 to 1.0 """ return self.parent._request('/labs/honeyscore/{}'.format(ip), {}) class Organization: def __init__(self, parent): self.parent = parent def add_member(self, user, notify=True): """Add the user to the organization. :param user: username or email address :type user: str :param notify: whether or not to send the user an email notification :type notify: bool :returns: True if it succeeded and raises an Exception otherwise """ return self.parent._request('/org/member/{}'.format(user), { 'notify': notify, }, method='PUT')['success'] def info(self): """Returns general information about the organization the current user is a member of. """ return self.parent._request('/org', {}) def remove_member(self, user): """Remove the user from the organization. :param user: username or email address :type user: str :returns: True if it succeeded and raises an Exception otherwise """ return self.parent._request('/org/member/{}'.format(user), {}, method='DELETE')['success'] def __init__(self, key, proxies=None): """Initializes the API object. :param key: The Shodan API key. :type key: str :param proxies: A proxies array for the requests library, e.g. {'https': 'your proxy'} :type proxies: dict """ self.api_key = key self.base_url = 'https://api.shodan.io' self.base_exploits_url = 'https://exploits.shodan.io' self.data = self.Data(self) self.dns = self.Dns(self) self.exploits = self.Exploits(self) self.labs = self.Labs(self) self.notifier = self.Notifier(self) self.org = self.Organization(self) self.tools = self.Tools(self) self.stream = Stream(key, proxies=proxies) self._session = requests.Session() if proxies: self._session.proxies.update(proxies) self._session.trust_env = False def _request(self, function, params, service='shodan', method='get'): """General-purpose function to create web requests to SHODAN. Arguments: function -- name of the function you want to execute params -- dictionary of parameters for the function Returns A dictionary containing the function's results. """ # Add the API key parameter automatically params['key'] = self.api_key # Determine the base_url based on which service we're interacting with base_url = { 'shodan': self.base_url, 'exploits': self.base_exploits_url, }.get(service, 'shodan') # Send the request try: method = method.lower() if method == 'post': data = self._session.post(base_url + function, params) elif method == 'put': data = self._session.put(base_url + function, params=params) elif method == 'delete': data = self._session.delete(base_url + function, params=params) else: data = self._session.get(base_url + function, params=params) except Exception: raise APIError('Unable to connect to Shodan') # Check that the API key wasn't rejected if data.status_code == 401: try: # Return the actual error message if the API returned valid JSON error = data.json()['error'] except Exception as e: # If the response looks like HTML then it's probably the 401 page that nginx returns # for 401 responses by default if data.text.startswith('<'): error = 'Invalid API key' else: # Otherwise lets raise the error message error = u'{}'.format(e) raise APIError(error) elif data.status_code == 403: raise APIError('Access denied (403 Forbidden)') # Parse the text into JSON try: data = data.json() except ValueError: raise APIError('Unable to parse JSON response') # Raise an exception if an error occurred if type(data) == dict and 'error' in data: raise APIError(data['error']) # Return the data return data def count(self, query, facets=None): """Returns the total number of search results for the query. :param query: Search query; identical syntax to the website :type query: str :param facets: (optional) A list of properties to get summary information on :type facets: str :returns: A dictionary with 1 main property: total. If facets have been provided then another property called "facets" will be available at the top-level of the dictionary. Visit the website for more detailed information. """ query_args = { 'query': query, } if facets: query_args['facets'] = create_facet_string(facets) return self._request('/shodan/host/count', query_args) def host(self, ips, history=False, minify=False): """Get all available information on an IP. :param ip: IP of the computer :type ip: str :param history: (optional) True if you want to grab the historical (non-current) banners for the host, False otherwise. :type history: bool :param minify: (optional) True to only return the list of ports and the general host information, no banners, False otherwise. :type minify: bool """ if isinstance(ips, basestring): ips = [ips] params = {} if history: params['history'] = history if minify: params['minify'] = minify return self._request('/shodan/host/%s' % ','.join(ips), params) def info(self): """Returns information about the current API key, such as a list of add-ons and other features that are enabled for the current user's API plan. """ return self._request('/api-info', {}) def ports(self): """Get a list of ports that Shodan crawls :returns: An array containing the ports that Shodan crawls for. """ return self._request('/shodan/ports', {}) def protocols(self): """Get a list of protocols that the Shodan on-demand scanning API supports. :returns: A dictionary containing the protocol name and description. """ return self._request('/shodan/protocols', {}) def scan(self, ips, force=False): """Scan a network using Shodan :param ips: A list of IPs or netblocks in CIDR notation or an object structured like: { "9.9.9.9": [ (443, "https"), (8080, "http") ], "1.1.1.0/24": [ (503, "modbus") ] } :type ips: str or dict :param force: Whether or not to force Shodan to re-scan the provided IPs. Only available to enterprise users. :type force: bool :returns: A dictionary with a unique ID to check on the scan progress, the number of IPs that will be crawled and how many scan credits are left. """ if isinstance(ips, basestring): ips = [ips] if isinstance(ips, dict): networks = json.dumps(ips) else: networks = ','.join(ips) params = { 'ips': networks, 'force': force, } return self._request('/shodan/scan', params, method='post') def scans(self, page=1): """Get a list of scans submitted :param page: Page through the list of scans 100 results at a time :type page: int """ return self._request('/shodan/scans', { 'page': page, }) def scan_internet(self, port, protocol): """Scan a network using Shodan :param port: The port that should get scanned. :type port: int :param port: The name of the protocol as returned by the protocols() method. :type port: str :returns: A dictionary with a unique ID to check on the scan progress. """ params = { 'port': port, 'protocol': protocol, } return self._request('/shodan/scan/internet', params, method='post') def scan_status(self, scan_id): """Get the status information about a previously submitted scan. :param id: The unique ID for the scan that was submitted :type id: str :returns: A dictionary with general information about the scan, including its status in getting processed. """ return self._request('/shodan/scan/%s' % scan_id, {}) def search(self, query, page=1, limit=None, offset=None, facets=None, minify=True): """Search the SHODAN database. :param query: Search query; identical syntax to the website :type query: str :param page: (optional) Page number of the search results :type page: int :param limit: (optional) Number of results to return :type limit: int :param offset: (optional) Search offset to begin getting results from :type offset: int :param facets: (optional) A list of properties to get summary information on :type facets: str :param minify: (optional) Whether to minify the banner and only return the important data :type minify: bool :returns: A dictionary with 2 main items: matches and total. If facets have been provided then another property called "facets" will be available at the top-level of the dictionary. Visit the website for more detailed information. """ args = { 'query': query, 'minify': minify, } if limit: args['limit'] = limit if offset: args['offset'] = offset else: args['page'] = page if facets: args['facets'] = create_facet_string(facets) return self._request('/shodan/host/search', args) def search_cursor(self, query, minify=True, retries=5): """Search the SHODAN database. This method returns an iterator that can directly be in a loop. Use it when you want to loop over all of the results of a search query. But this method doesn't return a "matches" array or the "total" information. And it also can't be used with facets, it's only use is to iterate over results more easily. :param query: Search query; identical syntax to the website :type query: str :param minify: (optional) Whether to minify the banner and only return the important data :type minify: bool :param retries: (optional) How often to retry the search in case it times out :type retries: int :returns: A search cursor that can be used as an iterator/ generator. """ page = 1 tries = 0 # Placeholder results object to make the while loop below easier results = { 'matches': [True], 'total': None, } while results['matches']: try: results = self.search(query, minify=minify, page=page) for banner in results['matches']: try: yield banner except GeneratorExit: return # exit out of the function page += 1 tries = 0 except Exception: # We've retried several times but it keeps failing, so lets error out if tries >= retries: raise APIError('Retry limit reached ({:d})'.format(retries)) tries += 1 time.sleep(tries) # wait (1 second * retry number) if the search errored out for some reason def search_facets(self): """Returns a list of search facets that can be used to get aggregate information about a search query. :returns: A list of strings where each is a facet name """ return self._request('/shodan/host/search/facets', {}) def search_filters(self): """Returns a list of search filters that are available. :returns: A list of strings where each is a filter name """ return self._request('/shodan/host/search/filters', {}) def search_tokens(self, query): """Returns information about the search query itself (filters used etc.) :param query: Search query; identical syntax to the website :type query: str :returns: A dictionary with 4 main properties: filters, errors, attributes and string. """ query_args = { 'query': query, } return self._request('/shodan/host/search/tokens', query_args) def services(self): """Get a list of services that Shodan crawls :returns: A dictionary containing the ports/ services that Shodan crawls for. The key is the port number and the value is the name of the service. """ return self._request('/shodan/services', {}) def queries(self, page=1, sort='timestamp', order='desc'): """List the search queries that have been shared by other users. :param page: Page number to iterate over results; each page contains 10 items :type page: int :param sort: Sort the list based on a property. Possible values are: votes, timestamp :type sort: str :param order: Whether to sort the list in ascending or descending order. Possible values are: asc, desc :type order: str :returns: A list of saved search queries (dictionaries). """ args = { 'page': page, 'sort': sort, 'order': order, } return self._request('/shodan/query', args) def queries_search(self, query, page=1): """Search the directory of saved search queries in Shodan. :param query: The search string to look for in the search query :type query: str :param page: Page number to iterate over results; each page contains 10 items :type page: int :returns: A list of saved search queries (dictionaries). """ args = { 'page': page, 'query': query, } return self._request('/shodan/query/search', args) def queries_tags(self, size=10): """Search the directory of saved search queries in Shodan. :param size: The number of tags to return :type size: int :returns: A list of tags. """ args = { 'size': size, } return self._request('/shodan/query/tags', args) def create_alert(self, name, ip, expires=0): """Create a network alert/ private firehose for the specified IP range(s) :param name: Name of the alert :type name: str :param ip: Network range(s) to monitor :type ip: str OR list of str :returns: A dict describing the alert """ data = { 'name': name, 'filters': { 'ip': ip, }, 'expires': expires, } response = api_request(self.api_key, '/shodan/alert', data=data, params={}, method='post', proxies=self._session.proxies) return response def edit_alert(self, aid, ip): """Edit the IPs that should be monitored by the alert. :param aid: Alert ID :type name: str :param ip: Network range(s) to monitor :type ip: str OR list of str :returns: A dict describing the alert """ data = { 'filters': { 'ip': ip, }, } response = api_request(self.api_key, '/shodan/alert/{}'.format(aid), data=data, params={}, method='post', proxies=self._session.proxies) return response def alerts(self, aid=None, include_expired=True): """List all of the active alerts that the user created.""" if aid: func = '/shodan/alert/%s/info' % aid else: func = '/shodan/alert/info' response = api_request(self.api_key, func, params={ 'include_expired': include_expired, }, proxies=self._session.proxies) return response def delete_alert(self, aid): """Delete the alert with the given ID.""" func = '/shodan/alert/%s' % aid response = api_request(self.api_key, func, params={}, method='delete', proxies=self._session.proxies) return response def alert_triggers(self): """Return a list of available triggers that can be enabled for alerts. :returns: A list of triggers """ return self._request('/shodan/alert/triggers', {}) def enable_alert_trigger(self, aid, trigger): """Enable the given trigger on the alert.""" return self._request('/shodan/alert/{}/trigger/{}'.format(aid, trigger), {}, method='put') def disable_alert_trigger(self, aid, trigger): """Disable the given trigger on the alert.""" return self._request('/shodan/alert/{}/trigger/{}'.format(aid, trigger), {}, method='delete') def ignore_alert_trigger_notification(self, aid, trigger, ip, port): """Ignore trigger notifications for the provided IP and port.""" return self._request('/shodan/alert/{}/trigger/{}/ignore/{}:{}'.format(aid, trigger, ip, port), {}, method='put') def unignore_alert_trigger_notification(self, aid, trigger, ip, port): """Re-enable trigger notifications for the provided IP and port""" return self._request('/shodan/alert/{}/trigger/{}/ignore/{}:{}'.format(aid, trigger, ip, port), {}, method='delete') def add_alert_notifier(self, aid, nid): """Enable the given notifier for an alert that has triggers enabled.""" return self._request('/shodan/alert/{}/notifier/{}'.format(aid, nid), {}, method='put') def remove_alert_notifier(self, aid, nid): """Remove the given notifier for an alert that has triggers enabled.""" return self._request('/shodan/alert/{}/notifier/{}'.format(aid, nid), {}, method='delete') shodan-1.26.1/shodan/stream.py0000664000175000017500000001421014172031036020055 0ustar achilleanachillean00000000000000import requests import json import ssl from .exception import APIError class Stream: base_url = 'https://stream.shodan.io' def __init__(self, api_key, proxies=None): self.api_key = api_key self.proxies = proxies def _create_stream(self, name, query=None, timeout=None): params = { 'key': self.api_key, } stream_url = self.base_url + name # The user doesn't want to use a timeout # If the timeout is specified as 0 then we also don't want to have a timeout if (timeout and timeout <= 0) or (timeout == 0): timeout = None # If the user requested a timeout then we need to disable heartbeat messages # which are intended to keep stream connections alive even if there isn't any data # flowing through. if timeout: params['heartbeat'] = False if query is not None: params['query'] = query try: while True: req = requests.get(stream_url, params=params, stream=True, timeout=timeout, proxies=self.proxies) # Status code 524 is special to Cloudflare # It means that no data was sent from the streaming servers which caused Cloudflare # to terminate the connection. # # We only want to exit if there was a timeout specified or the HTTP status code is # not specific to Cloudflare. if req.status_code != 524 or timeout >= 0: break except Exception: raise APIError('Unable to contact the Shodan Streaming API') if req.status_code != 200: try: data = json.loads(req.text) raise APIError(data['error']) except APIError: raise except Exception: pass raise APIError('Invalid API key or you do not have access to the Streaming API') if req.encoding is None: req.encoding = 'utf-8' return req def _iter_stream(self, stream, raw): for line in stream.iter_lines(): # The Streaming API sends out heartbeat messages that are newlines # We want to ignore those messages since they don't contain any data if line: if raw: yield line else: yield json.loads(line) def alert(self, aid=None, timeout=None, raw=False): if aid: stream = self._create_stream('/shodan/alert/%s' % aid, timeout=timeout) else: stream = self._create_stream('/shodan/alert', timeout=timeout) try: for line in self._iter_stream(stream, raw): yield line except requests.exceptions.ConnectionError: raise APIError('Stream timed out') except ssl.SSLError: raise APIError('Stream timed out') def asn(self, asn, raw=False, timeout=None): """ A filtered version of the "banners" stream to only return banners that match the ASNs of interest. :param asn: A list of ASN to return banner data on. :type asn: string[] """ stream = self._create_stream('/shodan/asn/%s' % ','.join(asn), timeout=timeout) for line in self._iter_stream(stream, raw): yield line def banners(self, raw=False, timeout=None): """A real-time feed of the data that Shodan is currently collecting. Note that this is only available to API subscription plans and for those it only returns a fraction of the data. """ stream = self._create_stream('/shodan/banners', timeout=timeout) for line in self._iter_stream(stream, raw): yield line def countries(self, countries, raw=False, timeout=None): """ A filtered version of the "banners" stream to only return banners that match the countries of interest. :param countries: A list of countries to return banner data on. :type countries: string[] """ stream = self._create_stream('/shodan/countries/%s' % ','.join(countries), timeout=timeout) for line in self._iter_stream(stream, raw): yield line def custom(self, query, raw=False, timeout=None): """ A filtered version of the "banners" stream to only return banners that match the query of interest. The query can vary and mix-match with different arguments (ports, tags, vulns, etc). :param query: A space-separated list of key:value filters query to return banner data on. :type query: string """ stream = self._create_stream('/shodan/custom', query=query, timeout=timeout) for line in self._iter_stream(stream, raw): yield line def ports(self, ports, raw=False, timeout=None): """ A filtered version of the "banners" stream to only return banners that match the ports of interest. :param ports: A list of ports to return banner data on. :type ports: int[] """ stream = self._create_stream('/shodan/ports/%s' % ','.join([str(port) for port in ports]), timeout=timeout) for line in self._iter_stream(stream, raw): yield line def tags(self, tags, raw=False, timeout=None): """ A filtered version of the "banners" stream to only return banners that match the tags of interest. :param tags: A list of tags to return banner data on. :type tags: string[] """ stream = self._create_stream('/shodan/tags/%s' % ','.join(tags), timeout=timeout) for line in self._iter_stream(stream, raw): yield line def vulns(self, vulns, raw=False, timeout=None): """ A filtered version of the "banners" stream to only return banners that match the vulnerabilities of interest. :param vulns: A list of vulns to return banner data on. :type vulns: string[] """ stream = self._create_stream('/shodan/vulns/%s' % ','.join(vulns), timeout=timeout) for line in self._iter_stream(stream, raw): yield line shodan-1.26.1/shodan/__init__.py0000664000175000017500000000010713335673020020326 0ustar achilleanachillean00000000000000from shodan.client import Shodan from shodan.exception import APIError shodan-1.26.1/shodan/helpers.py0000664000175000017500000001221013544172100020222 0ustar achilleanachillean00000000000000import gzip import requests import json from .exception import APIError try: basestring except NameError: basestring = str def create_facet_string(facets): """Converts a Python list of facets into a comma-separated string that can be understood by the Shodan API. """ facet_str = '' for facet in facets: if isinstance(facet, basestring): facet_str += facet else: facet_str += '{}:{}'.format(facet[0], facet[1]) facet_str += ',' return facet_str[:-1] def api_request(key, function, params=None, data=None, base_url='https://api.shodan.io', method='get', retries=1, proxies=None): """General-purpose function to create web requests to SHODAN. Arguments: function -- name of the function you want to execute params -- dictionary of parameters for the function proxies -- a proxies array for the requests library Returns A dictionary containing the function's results. """ # Add the API key parameter automatically params['key'] = key # Send the request tries = 0 error = False while tries <= retries: try: if method.lower() == 'post': data = requests.post(base_url + function, json.dumps(data), params=params, headers={'content-type': 'application/json'}, proxies=proxies) elif method.lower() == 'delete': data = requests.delete(base_url + function, params=params, proxies=proxies) elif method.lower() == 'put': data = requests.put(base_url + function, params=params, proxies=proxies) else: data = requests.get(base_url + function, params=params, proxies=proxies) # Exit out of the loop break except Exception: error = True tries += 1 if error and tries >= retries: raise APIError('Unable to connect to Shodan') # Check that the API key wasn't rejected if data.status_code == 401: try: raise APIError(data.json()['error']) except (ValueError, KeyError): pass raise APIError('Invalid API key') # Parse the text into JSON try: data = data.json() except Exception: raise APIError('Unable to parse JSON response') # Raise an exception if an error occurred if type(data) == dict and data.get('error', None): raise APIError(data['error']) # Return the data return data def iterate_files(files, fast=False): """Loop over all the records of the provided Shodan output file(s).""" loads = json.loads if fast: # Try to use ujson for parsing JSON if it's available and the user requested faster throughput # It's significantly faster at encoding/ decoding JSON but it doesn't support as # many options as the standard library. As such, we're mostly interested in using it for # decoding since reading/ parsing files will use up the most time. # pylint: disable=E0401 try: from ujson import loads except Exception: pass if isinstance(files, basestring): files = [files] for filename in files: # Create a file handle depending on the filetype if filename.endswith('.gz'): fin = gzip.open(filename, 'r') else: fin = open(filename, 'r') for line in fin: # Ensure the line has been decoded into a string to prevent errors w/ Python3 if not isinstance(line, basestring): line = line.decode('utf-8') # Convert the JSON into a native Python object banner = loads(line) yield banner def get_screenshot(banner): if 'opts' in banner and 'screenshot' in banner['opts']: return banner['opts']['screenshot'] return None def get_ip(banner): if 'ipv6' in banner: return banner['ipv6'] return banner['ip_str'] def open_file(filename, mode='a', compresslevel=9): return gzip.open(filename, mode, compresslevel) def write_banner(fout, banner): line = json.dumps(banner) + '\n' fout.write(line.encode('utf-8')) def humanize_bytes(bytes, precision=1): """Return a humanized string representation of a number of bytes. >>> humanize_bytes(1) '1 byte' >>> humanize_bytes(1024) '1.0 kB' >>> humanize_bytes(1024*123) '123.0 kB' >>> humanize_bytes(1024*12342) '12.1 MB' >>> humanize_bytes(1024*12342,2) '12.05 MB' >>> humanize_bytes(1024*1234,2) '1.21 MB' >>> humanize_bytes(1024*1234*1111,2) '1.31 GB' >>> humanize_bytes(1024*1234*1111,1) '1.3 GB' """ if bytes == 1: return '1 byte' if bytes < 1024: return '%.*f %s' % (precision, bytes, "bytes") suffixes = ['KB', 'MB', 'GB', 'TB', 'PB'] multiple = 1024.0 # .0 to force float on python 2 for suffix in suffixes: bytes /= multiple if bytes < multiple: return '%.*f %s' % (precision, bytes, suffix) return '%.*f %s' % (precision, bytes, suffix) shodan-1.26.1/shodan/exception.py0000664000175000017500000000042213430141555020564 0ustar achilleanachillean00000000000000class APIError(Exception): """This exception gets raised whenever a non-200 status code was returned by the Shodan API.""" def __init__(self, value): self.value = value def __str__(self): return self.value class APITimeout(APIError): pass shodan-1.26.1/shodan/threatnet.py0000664000175000017500000000417213430137525020574 0ustar achilleanachillean00000000000000import requests import json from .exception import APIError class Threatnet: """Wrapper around the Threatnet REST and Streaming APIs :param key: The Shodan API key that can be obtained from your account page (https://account.shodan.io) :type key: str :ivar stream: An instance of `shodan.Threatnet.Stream` that provides access to the Streaming API. """ class Stream: base_url = 'https://stream.shodan.io' def __init__(self, parent, proxies=None): self.parent = parent self.proxies = proxies def _create_stream(self, name): try: req = requests.get(self.base_url + name, params={'key': self.parent.api_key}, stream=True, proxies=self.proxies) except Exception: raise APIError('Unable to contact the Shodan Streaming API') if req.status_code != 200: try: raise APIError(req.json()['error']) except Exception: pass raise APIError('Invalid API key or you do not have access to the Streaming API') return req def events(self): stream = self._create_stream('/threatnet/events') for line in stream.iter_lines(): if line: banner = json.loads(line) yield banner def backscatter(self): stream = self._create_stream('/threatnet/backscatter') for line in stream.iter_lines(): if line: banner = json.loads(line) yield banner def activity(self): stream = self._create_stream('/threatnet/ssh') for line in stream.iter_lines(): if line: banner = json.loads(line) yield banner def __init__(self, key): """Initializes the API object. :param key: The Shodan API key. :type key: str """ self.api_key = key self.base_url = 'https://api.shodan.io' self.stream = self.Stream(self) shodan-1.26.1/shodan/__main__.py0000644000175000017500000007144214165663573020335 0ustar achilleanachillean00000000000000""" Shodan CLI Note: Always run "shodan init " before trying to execute any other command! A simple interface to search Shodan, download data and parse compressed JSON files. The following commands are currently supported: alert convert count data download honeyscore host info init myip parse radar scan search stats stream """ import click import csv import os import os.path import pkg_resources import shodan import shodan.helpers as helpers import threading import requests import time # The file converters that are used to go from .json.gz to various other formats from shodan.cli.converter import CsvConverter, KmlConverter, GeoJsonConverter, ExcelConverter, ImagesConverter # Constants from shodan.cli.settings import SHODAN_CONFIG_DIR, COLORIZE_FIELDS # Helper methods from shodan.cli.helpers import async_spinner, get_api_key, escape_data, timestr, open_streaming_file, get_banner_field, match_filters from shodan.cli.host import HOST_PRINT # Allow 3rd-parties to develop custom commands from click_plugins import with_plugins from pkg_resources import iter_entry_points # Large subcommands are stored in separate modules from shodan.cli.alert import alert from shodan.cli.data import data from shodan.cli.organization import org from shodan.cli.scan import scan # Make "-h" work like "--help" CONTEXT_SETTINGS = dict(help_option_names=['-h', '--help']) CONVERTERS = { 'kml': KmlConverter, 'csv': CsvConverter, 'geo.json': GeoJsonConverter, 'images': ImagesConverter, 'xlsx': ExcelConverter, } # Define a basestring type if necessary for Python3 compatibility try: basestring except NameError: basestring = str # Define the main entry point for all of our commands # and expose a way for 3rd-party plugins to tie into the Shodan CLI. @with_plugins(iter_entry_points('shodan.cli.plugins')) @click.group(context_settings=CONTEXT_SETTINGS) def main(): pass # Setup the large subcommands main.add_command(alert) main.add_command(data) main.add_command(org) main.add_command(scan) @main.command() @click.option('--fields', help='List of properties to output.', default=None) @click.argument('input', metavar='') @click.argument('format', metavar='', type=click.Choice(CONVERTERS.keys())) def convert(fields, input, format): """Convert the given input data file into a different format. The following file formats are supported: kml, csv, geo.json, images, xlsx Example: shodan convert data.json.gz kml """ # Check that the converter allows a custom list of fields converter_class = CONVERTERS.get(format) if fields: if not hasattr(converter_class, 'fields'): raise click.ClickException('File format doesnt support custom list of fields') converter_class.fields = [item.strip() for item in fields.split(',')] # Use the custom fields the user specified # Get the basename for the input file basename = input.replace('.json.gz', '').replace('.json', '') # Add the new file extension based on the format filename = '{}.{}'.format(basename, format) # Open the output file fout = open(filename, 'w') # Start a spinner finished_event = threading.Event() progress_bar_thread = threading.Thread(target=async_spinner, args=(finished_event,)) progress_bar_thread.start() # Initialize the file converter converter = converter_class(fout) converter.process([input]) finished_event.set() progress_bar_thread.join() if format == 'images': click.echo(click.style('\rSuccessfully extracted images to directory: {}'.format(converter.dirname), fg='green')) else: click.echo(click.style('\rSuccessfully created new file: {}'.format(filename), fg='green')) @main.command(name='domain') @click.argument('domain', metavar='') @click.option('--details', '-D', help='Lookup host information for any IPs in the domain results', default=False, is_flag=True) @click.option('--save', '-S', help='Save the information in the a file named after the domain (append if file exists).', default=False, is_flag=True) @click.option('--history', '-H', help='Include historical DNS data in the results', default=False, is_flag=True) @click.option('--type', '-T', help='Only returns DNS records of the provided type', default=None) def domain_info(domain, details, save, history, type): """View all available information for a domain""" key = get_api_key() api = shodan.Shodan(key) try: info = api.dns.domain_info(domain, history=history, type=type) except shodan.APIError as e: raise click.ClickException(e.value) # Grab the host information for any IP records that were returned hosts = {} if details: ips = [record['value'] for record in info['data'] if record['type'] in ['A', 'AAAA']] ips = set(ips) fout = None if save: filename = u'{}-hosts.json.gz'.format(domain) fout = helpers.open_file(filename) for ip in ips: try: hosts[ip] = api.host(ip) # Store the banners if requested if fout: for banner in hosts[ip]['data']: if 'placeholder' not in banner: helpers.write_banner(fout, banner) except shodan.APIError: pass # Ignore any API lookup errors as this isn't critical information # Save the DNS data if save: filename = u'{}.json.gz'.format(domain) fout = helpers.open_file(filename) for record in info['data']: helpers.write_banner(fout, record) click.secho(info['domain'].upper(), fg='green') click.echo('') for record in info['data']: click.echo( u'{:32} {:14} {}'.format( click.style(record['subdomain'], fg='cyan'), click.style(record['type'], fg='yellow'), record['value'] ), nl=False, ) if record['value'] in hosts: host = hosts[record['value']] click.secho(u' Ports: {}'.format(', '.join([str(port) for port in sorted(host['ports'])])), fg='blue', nl=False) click.echo('') @main.command() @click.argument('key', metavar='') def init(key): """Initialize the Shodan command-line""" # Create the directory if necessary shodan_dir = os.path.expanduser(SHODAN_CONFIG_DIR) if not os.path.isdir(shodan_dir): try: os.makedirs(shodan_dir) except OSError: raise click.ClickException('Unable to create directory to store the Shodan API key ({})'.format(shodan_dir)) # Make sure it's a valid API key key = key.strip() try: api = shodan.Shodan(key) api.info() except shodan.APIError as e: raise click.ClickException(e.value) # Store the API key in the user's directory keyfile = shodan_dir + '/api_key' with open(keyfile, 'w') as fout: fout.write(key.strip()) click.echo(click.style('Successfully initialized', fg='green')) os.chmod(keyfile, 0o600) @main.command() @click.argument('query', metavar='', nargs=-1) def count(query): """Returns the number of results for a search""" key = get_api_key() # Create the query string out of the provided tuple query = ' '.join(query).strip() # Make sure the user didn't supply an empty string if query == '': raise click.ClickException('Empty search query') # Perform the search api = shodan.Shodan(key) try: results = api.count(query) except shodan.APIError as e: raise click.ClickException(e.value) click.echo(results['total']) @main.command() @click.option('--limit', help='The number of results you want to download. -1 to download all the data possible.', default=1000, type=int) @click.argument('filename', metavar='') @click.argument('query', metavar='', nargs=-1) def download(limit, filename, query): """Download search results and save them in a compressed JSON file.""" key = get_api_key() # Create the query string out of the provided tuple query = ' '.join(query).strip() # Make sure the user didn't supply an empty string if query == '': raise click.ClickException('Empty search query') filename = filename.strip() if filename == '': raise click.ClickException('Empty filename') # Add the appropriate extension if it's not there atm if not filename.endswith('.json.gz'): filename += '.json.gz' # Perform the search api = shodan.Shodan(key) try: total = api.count(query)['total'] info = api.info() except Exception: raise click.ClickException('The Shodan API is unresponsive at the moment, please try again later.') # Print some summary information about the download request click.echo('Search query:\t\t\t%s' % query) click.echo('Total number of results:\t%s' % total) click.echo('Query credits left:\t\t%s' % info['unlocked_left']) click.echo('Output file:\t\t\t%s' % filename) if limit > total: limit = total # A limit of -1 means that we should download all the data if limit <= 0: limit = total with helpers.open_file(filename, 'w') as fout: count = 0 try: cursor = api.search_cursor(query, minify=False) with click.progressbar(cursor, length=limit) as bar: for banner in bar: helpers.write_banner(fout, banner) count += 1 if count >= limit: break except Exception: pass # Let the user know we're done if count < limit: click.echo(click.style('Notice: fewer results were saved than requested', 'yellow')) click.echo(click.style(u'Saved {} results into file {}'.format(count, filename), 'green')) @main.command() @click.option('--format', help='The output format for the host information. Possible values are: pretty, tsv.', default='pretty', type=click.Choice(['pretty', 'tsv'])) @click.option('--history', help='Show the complete history of the host.', default=False, is_flag=True) @click.option('--filename', '-O', help='Save the host information in the given file (append if file exists).', default=None) @click.option('--save', '-S', help='Save the host information in the a file named after the IP (append if file exists).', default=False, is_flag=True) @click.argument('ip', metavar='') def host(format, history, filename, save, ip): """View all available information for an IP address""" key = get_api_key() api = shodan.Shodan(key) try: host = api.host(ip, history=history) # Print the host information to the terminal using the user-specified format HOST_PRINT[format](host, history=history) # Store the results if filename or save: if save: filename = '{}.json.gz'.format(ip) # Add the appropriate extension if it's not there atm if not filename.endswith('.json.gz'): filename += '.json.gz' # Create/ append to the file fout = helpers.open_file(filename) for banner in sorted(host['data'], key=lambda k: k['port']): if 'placeholder' not in banner: helpers.write_banner(fout, banner) except shodan.APIError as e: raise click.ClickException(e.value) @main.command() def info(): """Shows general information about your account""" key = get_api_key() api = shodan.Shodan(key) try: results = api.info() except shodan.APIError as e: raise click.ClickException(e.value) click.echo("""Query credits available: {0} Scan credits available: {1} """.format(results['query_credits'], results['scan_credits'])) @main.command() @click.option('--color/--no-color', default=True) @click.option('--fields', help='List of properties to output.', default='ip_str,port,hostnames,data') @click.option('--filters', '-f', help='Filter the results for specific values using key:value pairs.', multiple=True) @click.option('--filename', '-O', help='Save the filtered results in the given file (append if file exists).') @click.option('--separator', help='The separator between the properties of the search results.', default=u'\t') @click.argument('filenames', metavar='', type=click.Path(exists=True), nargs=-1) def parse(color, fields, filters, filename, separator, filenames): """Extract information out of compressed JSON files.""" # Strip out any whitespace in the fields and turn them into an array fields = [item.strip() for item in fields.split(',')] if len(fields) == 0: raise click.ClickException('Please define at least one property to show') has_filters = len(filters) > 0 # Setup the output file handle fout = None if filename: # If no filters were provided raise an error since it doesn't make much sense w/out them if not has_filters: raise click.ClickException('Output file specified without any filters. Need to use filters with this option.') # Add the appropriate extension if it's not there atm if not filename.endswith('.json.gz'): filename += '.json.gz' fout = helpers.open_file(filename) for banner in helpers.iterate_files(filenames): row = u'' # Validate the banner against any provided filters if has_filters and not match_filters(banner, filters): continue # Append the data if fout: helpers.write_banner(fout, banner) # Loop over all the fields and print the banner as a row for i, field in enumerate(fields): tmp = u'' value = get_banner_field(banner, field) if value: field_type = type(value) # If the field is an array then merge it together if field_type == list: tmp = u';'.join(value) elif field_type in [int, float]: tmp = u'{}'.format(value) else: tmp = escape_data(value) # Colorize certain fields if the user wants it if color: tmp = click.style(tmp, fg=COLORIZE_FIELDS.get(field, 'white')) # Add the field information to the row if i > 0: row += separator row += tmp click.echo(row) @main.command() @click.option('--ipv6', '-6', is_flag=True, default=False, help='Try to use IPv6 instead of IPv4') def myip(ipv6): """Print your external IP address""" key = get_api_key() api = shodan.Shodan(key) # Use the IPv6-enabled domain if requested if ipv6: api.base_url = 'https://apiv6.shodan.io' try: click.echo(api.tools.myip()) except shodan.APIError as e: raise click.ClickException(e.value) @main.command() @click.option('--color/--no-color', default=True) @click.option('--fields', help='List of properties to show in the search results.', default='ip_str,port,hostnames,data') @click.option('--limit', help='The number of search results that should be returned. Maximum: 1000', default=100, type=int) @click.option('--separator', help='The separator between the properties of the search results.', default='\t') @click.argument('query', metavar='', nargs=-1) def search(color, fields, limit, separator, query): """Search the Shodan database""" key = get_api_key() # Create the query string out of the provided tuple query = ' '.join(query).strip() # Make sure the user didn't supply an empty string if query == '': raise click.ClickException('Empty search query') # For now we only allow up to 1000 results at a time if limit > 1000: raise click.ClickException('Too many results requested, maximum is 1,000') # Strip out any whitespace in the fields and turn them into an array fields = [item.strip() for item in fields.split(',')] if len(fields) == 0: raise click.ClickException('Please define at least one property to show') # Perform the search api = shodan.Shodan(key) try: results = api.search(query, limit=limit) except shodan.APIError as e: raise click.ClickException(e.value) # Error out if no results were found if results['total'] == 0: raise click.ClickException('No search results found') # We buffer the entire output so we can use click's pager functionality output = u'' for banner in results['matches']: row = u'' # Loop over all the fields and print the banner as a row for field in fields: tmp = u'' value = get_banner_field(banner, field) if value: field_type = type(value) # If the field is an array then merge it together if field_type == list: tmp = u';'.join(value) elif field_type in [int, float]: tmp = u'{}'.format(value) else: tmp = escape_data(value) # Colorize certain fields if the user wants it if color: tmp = click.style(tmp, fg=COLORIZE_FIELDS.get(field, 'white')) # Add the field information to the row row += tmp row += separator # click.echo(out + separator, nl=False) output += row + u'\n' # click.echo('') click.echo_via_pager(output) @main.command() @click.option('--limit', help='The number of results to return.', default=10, type=int) @click.option('--facets', help='List of facets to get statistics for.', default='country,org') @click.option('--filename', '-O', help='Save the results in a CSV file of the provided name.', default=None) @click.argument('query', metavar='', nargs=-1) def stats(limit, facets, filename, query): """Provide summary information about a search query""" # Setup Shodan key = get_api_key() api = shodan.Shodan(key) # Create the query string out of the provided tuple query = ' '.join(query).strip() # Make sure the user didn't supply an empty string if query == '': raise click.ClickException('Empty search query') facets = facets.split(',') facets = [(facet, limit) for facet in facets] # Perform the search try: results = api.count(query, facets=facets) except shodan.APIError as e: raise click.ClickException(e.value) # Print the stats tables for facet in results['facets']: click.echo('Top {} Results for Facet: {}'.format(len(results['facets'][facet]), facet)) for item in results['facets'][facet]: # Force the value to be a string - necessary because some facet values are numbers value = u'{}'.format(item['value']) click.echo(click.style(u'{:28s}'.format(value), fg='cyan'), nl=False) click.echo(click.style(u'{:12,d}'.format(item['count']), fg='green')) click.echo('') # Create the output file if requested fout = None if filename: if not filename.endswith('.csv'): filename += '.csv' fout = open(filename, 'w') writer = csv.writer(fout, dialect=csv.excel) # Write the header writer.writerow(['Query', query]) # Add an empty line to separate rows writer.writerow([]) # Write the header that contains the facets row = [] for facet in results['facets']: row.append(facet) row.append('') writer.writerow(row) # Every facet has 2 columns (key, value) counter = 0 has_items = True while has_items: # pylint: disable=W0612 row = ['' for i in range(len(results['facets']) * 2)] pos = 0 has_items = False for facet in results['facets']: values = results['facets'][facet] # Add the values for the facet into the current row if len(values) > counter: has_items = True row[pos] = values[counter]['value'] row[pos + 1] = values[counter]['count'] pos += 2 # Write out the row if has_items: writer.writerow(row) # Move to the next row of values counter += 1 @main.command() @click.option('--streamer', help='Specify a custom Shodan stream server to use for grabbing data.', default='https://stream.shodan.io', type=str) @click.option('--fields', help='List of properties to output.', default='ip_str,port,hostnames,data') @click.option('--separator', help='The separator between the properties of the search results.', default='\t') @click.option('--datadir', help='Save the stream data into the specified directory as .json.gz files.', default=None, type=str) @click.option('--asn', help='A comma-separated list of ASNs to grab data on.', default=None, type=str) @click.option('--alert', help='The network alert ID or "all" to subscribe to all network alerts on your account.', default=None, type=str) @click.option('--countries', help='A comma-separated list of countries to grab data on.', default=None, type=str) @click.option('--custom-filters', help='A space-separated list of filters query to grab data on.', default=None, type=str) @click.option('--ports', help='A comma-separated list of ports to grab data on.', default=None, type=str) @click.option('--tags', help='A comma-separated list of tags to grab data on.', default=None, type=str) @click.option('--vulns', help='A comma-separated list of vulnerabilities to grab data on.', default=None, type=str) @click.option('--limit', help='The number of results you want to download. -1 to download all the data possible.', default=-1, type=int) @click.option('--compresslevel', help='The gzip compression level (0-9; 0 = no compression, 9 = most compression', default=9, type=int) @click.option('--timeout', help='Timeout. Should the shodan stream cease to send data, then timeout after seconds.', default=0, type=int) @click.option('--color/--no-color', default=True) @click.option('--quiet', help='Disable the printing of information to the screen.', is_flag=True) def stream(streamer, fields, separator, datadir, asn, alert, countries, custom_filters, ports, tags, vulns, limit, compresslevel, timeout, color, quiet): """Stream data in real-time.""" # Setup the Shodan API key = get_api_key() api = shodan.Shodan(key) # Temporarily change the baseurl api.stream.base_url = streamer # Strip out any whitespace in the fields and turn them into an array fields = [item.strip() for item in fields.split(',')] if len(fields) == 0: raise click.ClickException('Please define at least one property to show') # The user must choose "ports", "countries", "asn" or nothing - can't select multiple # filtered streams at once. stream_type = [] if ports: stream_type.append('ports') if countries: stream_type.append('countries') if asn: stream_type.append('asn') if alert: stream_type.append('alert') if tags: stream_type.append('tags') if vulns: stream_type.append('vulns') if custom_filters: stream_type.append('custom_filters') if len(stream_type) > 1: raise click.ClickException('Please use --ports, --countries, --custom, --tags, --vulns OR --asn. You cant subscribe to multiple filtered streams at once.') stream_args = None # Turn the list of ports into integers if ports: try: stream_args = [int(item.strip()) for item in ports.split(',')] except ValueError: raise click.ClickException('Invalid list of ports') if alert: alert = alert.strip() if alert.lower() != 'all': stream_args = alert if asn: stream_args = asn.split(',') if countries: stream_args = countries.split(',') if tags: stream_args = tags.split(',') if vulns: stream_args = vulns.split(',') if custom_filters: stream_args = custom_filters # Flatten the list of stream types # Possible values are: # - all # - asn # - countries # - ports if len(stream_type) == 1: stream_type = stream_type[0] else: stream_type = 'all' # Decide which stream to subscribe to based on whether or not ports were selected def _create_stream(name, args, timeout): return { 'all': api.stream.banners(timeout=timeout), 'alert': api.stream.alert(args, timeout=timeout), 'asn': api.stream.asn(args, timeout=timeout), 'countries': api.stream.countries(args, timeout=timeout), 'custom_filters': api.stream.custom(args, timeout=timeout), 'ports': api.stream.ports(args, timeout=timeout), 'tags': api.stream.tags(args, timeout=timeout), 'vulns': api.stream.vulns(args, timeout=timeout), }.get(name, 'all') stream = _create_stream(stream_type, stream_args, timeout=timeout) counter = 0 quit = False last_time = timestr() fout = None if datadir: fout = open_streaming_file(datadir, last_time, compresslevel) while not quit: try: for banner in stream: # Limit the number of results to output if limit > 0: counter += 1 if counter > limit: quit = True break # Write the data to the file if datadir: cur_time = timestr() if cur_time != last_time: last_time = cur_time fout.close() fout = open_streaming_file(datadir, last_time) helpers.write_banner(fout, banner) # Print the banner information to stdout if not quiet: row = u'' # Loop over all the fields and print the banner as a row for field in fields: tmp = u'' value = get_banner_field(banner, field) if value: field_type = type(value) # If the field is an array then merge it together if field_type == list: tmp = u';'.join(value) elif field_type in [int, float]: tmp = u'{}'.format(value) else: tmp = escape_data(value) # Colorize certain fields if the user wants it if color: tmp = click.style(tmp, fg=COLORIZE_FIELDS.get(field, 'white')) # Add the field information to the row row += tmp row += separator click.echo(row) except requests.exceptions.Timeout: raise click.ClickException('Connection timed out') except KeyboardInterrupt: quit = True except shodan.APIError as e: raise click.ClickException(e.value) except Exception: # For other errors lets just wait a bit and try to reconnect again time.sleep(1) # Create a new stream object to subscribe to stream = _create_stream(stream_type, stream_args, timeout=timeout) @main.command() @click.argument('ip', metavar='') def honeyscore(ip): """Check whether the IP is a honeypot or not.""" key = get_api_key() api = shodan.Shodan(key) try: score = api.labs.honeyscore(ip) if score == 1.0: click.echo(click.style('Honeypot detected', fg='red')) elif score > 0.5: click.echo(click.style('Probably a honeypot', fg='yellow')) else: click.echo(click.style('Not a honeypot', fg='green')) click.echo('Score: {}'.format(score)) except Exception: raise click.ClickException('Unable to calculate honeyscore') @main.command() def radar(): """Real-Time Map of some results as Shodan finds them.""" key = get_api_key() api = shodan.Shodan(key) from shodan.cli.worldmap import launch_map try: launch_map(api) except shodan.APIError as e: raise click.ClickException(e.value) except Exception as e: raise click.ClickException(u'{}'.format(e)) @main.command() def version(): """Print version of this tool.""" print(pkg_resources.get_distribution("shodan").version) if __name__ == '__main__': main() shodan-1.26.1/docs/0000775000175000017500000000000014172031225015666 5ustar achilleanachillean00000000000000shodan-1.26.1/docs/examples/0000775000175000017500000000000014172031225017504 5ustar achilleanachillean00000000000000shodan-1.26.1/docs/examples/gifcreator.rst0000664000175000017500000000732113634247043022377 0ustar achilleanachillean00000000000000GIF Creator ----------- Shodan keeps a full history of all the information that has been gathered on an IP address. With the API, you're able to retrieve that history and we're going to use that to create a tool that outputs GIFs made of the screenshots that the Shodan crawlers gather. The below code requires the following Python packages: - arrow - shodan The **arrow** package is used to parse the *timestamp* field of the banner into a Python `datetime` object. In addition to the above Python packages, you also need to have the **ImageMagick** software installed. If you're working on Ubuntu or another distro using **apt** you can run the following command: .. code-block:: bash sudo apt-get install imagemagick This will provide us with the **convert** command which is needed to merge several images into an animated GIF. There are a few key Shodan methods/ parameters that make the script work: 1. :py:func:`shodan.helpers.iterate_files()` to loop through the Shodan data file 2. **history** flag on the :py:func:`shodan.Shodan.host` method to get all the banners for an IP that Shodan has collected over the years .. code-block:: python #!/usr/bin/env python # gifcreator.py # # Dependencies: # - arrow # - shodan # # Installation: # sudo easy_install arrow shodan # sudo apt-get install imagemagick # # Usage: # 1. Download a json.gz file using the website or the Shodan command-line tool (https://cli.shodan.io). # For example: # shodan download screenshots.json.gz has_screenshot:true # 2. Run the tool on the file: # python gifcreator.py screenshots.json.gz import arrow import os import shodan import shodan.helpers as helpers import sys # Settings API_KEY = '' MIN_SCREENS = 5 # Number of screenshots that Shodan needs to have in order to make a GIF MAX_SCREENS = 24 if len(sys.argv) != 2: print('Usage: {} '.format(sys.argv[0])) sys.exit(1) # GIFs are stored in the local "data" directory os.mkdir('data') # We need to connect to the API to lookup the historical host information api = shodan.Shodan(API_KEY) # Use the shodan.helpers.iterate_files() method to loop over the Shodan data file for result in helpers.iterate_files(sys.argv[1]): # Get the historic info host = api.host(result['ip_str'], history=True) # Count how many screenshots this host has screenshots = [] for banner in host['data']: # Extract the image from the banner data if 'opts' in banner and 'screenshot' in banner['opts']: # Sort the images by the time they were collected so the GIF will loop # based on the local time regardless of which day the banner was taken. timestamp = arrow.get(banner['timestamp']).time() sort_key = timestamp.hour screenshots.append(( sort_key, banner['opts']['screenshot']['data'] )) # Ignore any further screenshots if we already have MAX_SCREENS number of images if len(screenshots) >= MAX_SCREENS: break # Extract the screenshots and turn them into a GIF if we've got the necessary # amount of images. if len(screenshots) >= MIN_SCREENS: for (i, screenshot) in enumerate(sorted(screenshots, key=lambda x: x[0], reverse=True)): open('/tmp/gif-image-{}.jpg'.format(i), 'w').write(screenshot[1].decode('base64')) # Create the actual GIF using the ImageMagick "convert" command os.system('convert -layers OptimizePlus -delay 5x10 /tmp/gif-image-*.jpg -loop 0 +dither -colors 256 -depth 8 data/{}.gif'.format(result['ip_str'])) # Clean up the temporary files os.system('rm -f /tmp/gif-image-*.jpg') # Show a progress indicator print(result['ip_str']) The full code is also available on GitHub: https://gist.github.com/achillean/963eea552233d9550101 shodan-1.26.1/docs/examples/cert-stream.rst0000664000175000017500000000314713563035251022477 0ustar achilleanachillean00000000000000Access SSL certificates in Real-Time ------------------------------------ The new Shodan Streaming API provides real-time access to the information that Shodan is gathering at the moment. Using the Streaming API, you get the raw access to potentially all the data that ends up in the Shodan search engine. Note that you can't search with the Streaming API or perform any other operations that you're accustomed to with the REST API. This is meant for large-scale consumption of real-time data. This script only works with people that have a subscription API plan! And by default the Streaming API only returns 1% of the data that Shodan gathers. If you wish to have more access please contact us at support@shodan.io for pricing information. .. code-block:: python #!/usr/bin/env python # # cert-stream.py # Stream the SSL certificates that Shodan is collecting at the moment # # WARNING: This script only works with people that have a subscription API plan! # And by default the Streaming API only returns 1% of the data that Shodan gathers. # If you wish to have more access please contact us at sales@shodan.io for pricing # information. # # Author: achillean import shodan import sys # Configuration API_KEY = 'YOUR API KEY' try: # Setup the api api = shodan.Shodan(API_KEY) print('Listening for certs...') for banner in api.stream.ports([443, 8443]): if 'ssl' in banner: # Print out all the SSL information that Shodan has collected print(banner['ssl']) except Exception as e: print('Error: {}'.format(e)) sys.exit(1) shodan-1.26.1/docs/examples/query-summary.rst0000664000175000017500000000642413634247043023115 0ustar achilleanachillean00000000000000Collecting Summary Information using Facets ------------------------------------------- A powerful ability of the Shodan API is to get summary information on a variety of properties. For example, if you wanted to learn which countries have the most Apache servers then you would use facets. If you wanted to figure out which version of nginx is most popular, you would use facets. Or if you wanted to see what the uptime distribution is for Microsoft-IIS servers then you would use facets. The following script shows how to use the `shodan.Shodan.count()` method to search Shodan without returning any results as well as asking the API to return faceted information on the organization, domain, port, ASN and country. .. code-block:: python #!/usr/bin/env python # # query-summary.py # Search Shodan and print summary information for the query. # # Author: achillean import shodan import sys # Configuration API_KEY = 'YOUR API KEY' # The list of properties we want summary information on FACETS = [ 'org', 'domain', 'port', 'asn', # We only care about the top 3 countries, this is how we let Shodan know to return 3 instead of the # default 5 for a facet. If you want to see more than 5, you could do ('country', 1000) for example # to see the top 1,000 countries for a search query. ('country', 3), ] FACET_TITLES = { 'org': 'Top 5 Organizations', 'domain': 'Top 5 Domains', 'port': 'Top 5 Ports', 'asn': 'Top 5 Autonomous Systems', 'country': 'Top 3 Countries', } # Input validation if len(sys.argv) == 1: print('Usage: %s ' % sys.argv[0]) sys.exit(1) try: # Setup the api api = shodan.Shodan(API_KEY) # Generate a query string out of the command-line arguments query = ' '.join(sys.argv[1:]) # Use the count() method because it doesn't return results and doesn't require a paid API plan # And it also runs faster than doing a search(). result = api.count(query, facets=FACETS) print('Shodan Summary Information') print('Query: %s' % query) print('Total Results: %s\n' % result['total']) # Print the summary info from the facets for facet in result['facets']: print(FACET_TITLES[facet]) for term in result['facets'][facet]: print('%s: %s' % (term['value'], term['count'])) # Print an empty line between summary info print('') except Exception as e: print('Error: %s' % e) sys.exit(1) """ Sample Output ============= ./query-summary.py apache Shodan Summary Information Query: apache Total Results: 34612043 Top 5 Organizations Amazon.com: 808061 Ecommerce Corporation: 788704 Verio Web Hosting: 760112 Unified Layer: 627827 GoDaddy.com, LLC: 567004 Top 5 Domains secureserver.net: 562047 unifiedlayer.com: 494399 t-ipconnect.de: 385792 netart.pl: 194817 wanadoo.fr: 151925 Top 5 Ports 80: 24118703 443: 8330932 8080: 1479050 81: 359025 8443: 231441 Top 5 Autonomous Systems as32392: 580002 as2914: 465786 as26496: 414998 as48030: 332000 as8560: 255774 Top 3 Countries US: 13227366 DE: 2900530 JP: 2014506 """ shodan-1.26.1/docs/examples/basic-search.rst0000664000175000017500000000133212646577202022577 0ustar achilleanachillean00000000000000Basic Shodan Search ------------------- .. code-block:: python #!/usr/bin/env python # # shodan_ips.py # Search SHODAN and print a list of IPs matching the query # # Author: achillean import shodan import sys # Configuration API_KEY = "YOUR_API_KEY" # Input validation if len(sys.argv) == 1: print 'Usage: %s ' % sys.argv[0] sys.exit(1) try: # Setup the api api = shodan.Shodan(API_KEY) # Perform the search query = ' '.join(sys.argv[1:]) result = api.search(query) # Loop through the matches and print each IP for service in result['matches']: print service['ip_str'] except Exception as e: print 'Error: %s' % e sys.exit(1)shodan-1.26.1/docs/make.bat0000664000175000017500000001507312414103244017277 0ustar achilleanachillean00000000000000@ECHO OFF REM Command file for Sphinx documentation if "%SPHINXBUILD%" == "" ( set SPHINXBUILD=sphinx-build ) set BUILDDIR=_build set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% . set I18NSPHINXOPTS=%SPHINXOPTS% . if NOT "%PAPER%" == "" ( set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS% set I18NSPHINXOPTS=-D latex_paper_size=%PAPER% %I18NSPHINXOPTS% ) if "%1" == "" goto help if "%1" == "help" ( :help echo.Please use `make ^` where ^ is one of echo. html to make standalone HTML files echo. dirhtml to make HTML files named index.html in directories echo. singlehtml to make a single large HTML file echo. pickle to make pickle files echo. json to make JSON files echo. htmlhelp to make HTML files and a HTML help project echo. qthelp to make HTML files and a qthelp project echo. devhelp to make HTML files and a Devhelp project echo. epub to make an epub echo. latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter echo. text to make text files echo. man to make manual pages echo. texinfo to make Texinfo files echo. gettext to make PO message catalogs echo. changes to make an overview over all changed/added/deprecated items echo. xml to make Docutils-native XML files echo. pseudoxml to make pseudoxml-XML files for display purposes echo. linkcheck to check all external links for integrity echo. doctest to run all doctests embedded in the documentation if enabled goto end ) if "%1" == "clean" ( for /d %%i in (%BUILDDIR%\*) do rmdir /q /s %%i del /q /s %BUILDDIR%\* goto end ) %SPHINXBUILD% 2> nul if errorlevel 9009 ( echo. echo.The 'sphinx-build' command was not found. Make sure you have Sphinx echo.installed, then set the SPHINXBUILD environment variable to point echo.to the full path of the 'sphinx-build' executable. Alternatively you echo.may add the Sphinx directory to PATH. echo. echo.If you don't have Sphinx installed, grab it from echo.http://sphinx-doc.org/ exit /b 1 ) if "%1" == "html" ( %SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/html. goto end ) if "%1" == "dirhtml" ( %SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml. goto end ) if "%1" == "singlehtml" ( %SPHINXBUILD% -b singlehtml %ALLSPHINXOPTS% %BUILDDIR%/singlehtml if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/singlehtml. goto end ) if "%1" == "pickle" ( %SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can process the pickle files. goto end ) if "%1" == "json" ( %SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can process the JSON files. goto end ) if "%1" == "htmlhelp" ( %SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can run HTML Help Workshop with the ^ .hhp project file in %BUILDDIR%/htmlhelp. goto end ) if "%1" == "qthelp" ( %SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can run "qcollectiongenerator" with the ^ .qhcp project file in %BUILDDIR%/qthelp, like this: echo.^> qcollectiongenerator %BUILDDIR%\qthelp\shodan-python.qhcp echo.To view the help file: echo.^> assistant -collectionFile %BUILDDIR%\qthelp\shodan-python.ghc goto end ) if "%1" == "devhelp" ( %SPHINXBUILD% -b devhelp %ALLSPHINXOPTS% %BUILDDIR%/devhelp if errorlevel 1 exit /b 1 echo. echo.Build finished. goto end ) if "%1" == "epub" ( %SPHINXBUILD% -b epub %ALLSPHINXOPTS% %BUILDDIR%/epub if errorlevel 1 exit /b 1 echo. echo.Build finished. The epub file is in %BUILDDIR%/epub. goto end ) if "%1" == "latex" ( %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex if errorlevel 1 exit /b 1 echo. echo.Build finished; the LaTeX files are in %BUILDDIR%/latex. goto end ) if "%1" == "latexpdf" ( %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex cd %BUILDDIR%/latex make all-pdf cd %BUILDDIR%/.. echo. echo.Build finished; the PDF files are in %BUILDDIR%/latex. goto end ) if "%1" == "latexpdfja" ( %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex cd %BUILDDIR%/latex make all-pdf-ja cd %BUILDDIR%/.. echo. echo.Build finished; the PDF files are in %BUILDDIR%/latex. goto end ) if "%1" == "text" ( %SPHINXBUILD% -b text %ALLSPHINXOPTS% %BUILDDIR%/text if errorlevel 1 exit /b 1 echo. echo.Build finished. The text files are in %BUILDDIR%/text. goto end ) if "%1" == "man" ( %SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man if errorlevel 1 exit /b 1 echo. echo.Build finished. The manual pages are in %BUILDDIR%/man. goto end ) if "%1" == "texinfo" ( %SPHINXBUILD% -b texinfo %ALLSPHINXOPTS% %BUILDDIR%/texinfo if errorlevel 1 exit /b 1 echo. echo.Build finished. The Texinfo files are in %BUILDDIR%/texinfo. goto end ) if "%1" == "gettext" ( %SPHINXBUILD% -b gettext %I18NSPHINXOPTS% %BUILDDIR%/locale if errorlevel 1 exit /b 1 echo. echo.Build finished. The message catalogs are in %BUILDDIR%/locale. goto end ) if "%1" == "changes" ( %SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes if errorlevel 1 exit /b 1 echo. echo.The overview file is in %BUILDDIR%/changes. goto end ) if "%1" == "linkcheck" ( %SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck if errorlevel 1 exit /b 1 echo. echo.Link check complete; look for any errors in the above output ^ or in %BUILDDIR%/linkcheck/output.txt. goto end ) if "%1" == "doctest" ( %SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest if errorlevel 1 exit /b 1 echo. echo.Testing of doctests in the sources finished, look at the ^ results in %BUILDDIR%/doctest/output.txt. goto end ) if "%1" == "xml" ( %SPHINXBUILD% -b xml %ALLSPHINXOPTS% %BUILDDIR%/xml if errorlevel 1 exit /b 1 echo. echo.Build finished. The XML files are in %BUILDDIR%/xml. goto end ) if "%1" == "pseudoxml" ( %SPHINXBUILD% -b pseudoxml %ALLSPHINXOPTS% %BUILDDIR%/pseudoxml if errorlevel 1 exit /b 1 echo. echo.Build finished. The pseudo-XML files are in %BUILDDIR%/pseudoxml. goto end ) :end shodan-1.26.1/docs/api.rst0000664000175000017500000000044112414103244017166 0ustar achilleanachillean00000000000000.. _api: shodan ====== .. module:: shodan .. autoclass:: Shodan :inherited-members: .. autoclass:: shodan::Shodan.Exploits :inherited-members: .. autoclass:: shodan::Shodan.Stream :inherited-members: Exceptions ~~~~~~~~~~ .. autoexception:: shodan.APIErrorshodan-1.26.1/docs/index.rst0000664000175000017500000000214313253034257017536 0ustar achilleanachillean00000000000000.. shodan-python documentation master file, created by sphinx-quickstart on Thu Jan 23 00:56:29 2014. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. shodan - The official Python library for the Shodan search engine ================================================================= This is the official Python wrapper around both the Shodan REST API as well as the experimental Streaming API. And as a bonus it also lets you search for exploits using the Shodan Exploits REST API. If you're not sure where to start simply go through the "Getting Started" section of the documentation and work your way down through the examples. For more information about Shodan and how to use the API please visit our official help center at: https://help.shodan.io Introduction ~~~~~~~~~~~~ .. toctree:: :maxdepth: 2 tutorial Examples ~~~~~~~~ .. toctree:: :maxdepth: 2 examples/basic-search examples/query-summary examples/cert-stream examples/gifcreator API Reference ~~~~~~~~~~~~~ .. toctree:: :maxdepth: 2 api shodan-1.26.1/docs/Makefile0000664000175000017500000001520612414103244017330 0ustar achilleanachillean00000000000000# Makefile for Sphinx documentation # # You can set these variables from the command line. SPHINXOPTS = SPHINXBUILD = sphinx-build PAPER = BUILDDIR = _build # User-friendly check for sphinx-build ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1) $(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/) endif # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . # the i18n builder cannot share the environment and doctrees with the others I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext help: @echo "Please use \`make ' where is one of" @echo " html to make standalone HTML files" @echo " dirhtml to make HTML files named index.html in directories" @echo " singlehtml to make a single large HTML file" @echo " pickle to make pickle files" @echo " json to make JSON files" @echo " htmlhelp to make HTML files and a HTML help project" @echo " qthelp to make HTML files and a qthelp project" @echo " devhelp to make HTML files and a Devhelp project" @echo " epub to make an epub" @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" @echo " latexpdf to make LaTeX files and run them through pdflatex" @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" @echo " text to make text files" @echo " man to make manual pages" @echo " texinfo to make Texinfo files" @echo " info to make Texinfo files and run them through makeinfo" @echo " gettext to make PO message catalogs" @echo " changes to make an overview of all changed/added/deprecated items" @echo " xml to make Docutils-native XML files" @echo " pseudoxml to make pseudoxml-XML files for display purposes" @echo " linkcheck to check all external links for integrity" @echo " doctest to run all doctests embedded in the documentation (if enabled)" clean: rm -rf $(BUILDDIR)/* html: $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." dirhtml: $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." singlehtml: $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml @echo @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." pickle: $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle @echo @echo "Build finished; now you can process the pickle files." json: $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json @echo @echo "Build finished; now you can process the JSON files." htmlhelp: $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp @echo @echo "Build finished; now you can run HTML Help Workshop with the" \ ".hhp project file in $(BUILDDIR)/htmlhelp." qthelp: $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp @echo @echo "Build finished; now you can run "qcollectiongenerator" with the" \ ".qhcp project file in $(BUILDDIR)/qthelp, like this:" @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/shodan-python.qhcp" @echo "To view the help file:" @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/shodan-python.qhc" devhelp: $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp @echo @echo "Build finished." @echo "To view the help file:" @echo "# mkdir -p $$HOME/.local/share/devhelp/shodan-python" @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/shodan-python" @echo "# devhelp" epub: $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub @echo @echo "Build finished. The epub file is in $(BUILDDIR)/epub." latex: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." @echo "Run \`make' in that directory to run these through (pdf)latex" \ "(use \`make latexpdf' here to do that automatically)." latexpdf: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through pdflatex..." $(MAKE) -C $(BUILDDIR)/latex all-pdf @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." latexpdfja: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through platex and dvipdfmx..." $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." text: $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text @echo @echo "Build finished. The text files are in $(BUILDDIR)/text." man: $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man @echo @echo "Build finished. The manual pages are in $(BUILDDIR)/man." texinfo: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." @echo "Run \`make' in that directory to run these through makeinfo" \ "(use \`make info' here to do that automatically)." info: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo "Running Texinfo files through makeinfo..." make -C $(BUILDDIR)/texinfo info @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." gettext: $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale @echo @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." changes: $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes @echo @echo "The overview file is in $(BUILDDIR)/changes." linkcheck: $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck @echo @echo "Link check complete; look for any errors in the above output " \ "or in $(BUILDDIR)/linkcheck/output.txt." doctest: $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest @echo "Testing of doctests in the sources finished, look at the " \ "results in $(BUILDDIR)/doctest/output.txt." xml: $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml @echo @echo "Build finished. The XML files are in $(BUILDDIR)/xml." pseudoxml: $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml @echo @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." shodan-1.26.1/docs/conf.py0000664000175000017500000002011012414103244017155 0ustar achilleanachillean00000000000000# -*- coding: utf-8 -*- # # shodan-python documentation build configuration file, created by # sphinx-quickstart on Thu Jan 23 00:56:29 2014. # # This file is execfile()d with the current directory set to its # containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. import sys import os # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. #sys.path.insert(0, os.path.abspath('.')) # -- General configuration ------------------------------------------------ # If your documentation needs a minimal Sphinx version, state it here. #needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = [ 'sphinx.ext.autodoc', 'sphinx.ext.todo', 'sphinx.ext.viewcode', ] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The suffix of source filenames. source_suffix = '.rst' # The encoding of source files. #source_encoding = 'utf-8-sig' # The master toctree document. master_doc = 'index' # General information about the project. project = u'shodan-python' copyright = u'2014, achillean' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. version = '1.0' # The full version, including alpha/beta/rc tags. release = '1.0' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. #language = None # There are two options for replacing |today|: either, you set today to some # non-false value, then it is used: #today = '' # Else, today_fmt is used as the format for a strftime call. #today_fmt = '%B %d, %Y' # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. exclude_patterns = ['_build'] # The reST default role (used for this markup: `text`) to use for all # documents. #default_role = None # If true, '()' will be appended to :func: etc. cross-reference text. #add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). #add_module_names = True # If true, sectionauthor and moduleauthor directives will be shown in the # output. They are ignored by default. #show_authors = False # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # A list of ignored prefixes for module index sorting. #modindex_common_prefix = [] # If true, keep warnings as "system message" paragraphs in the built documents. #keep_warnings = False # -- Options for HTML output ---------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. html_theme = 'default' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. #html_theme_options = {} # Add any paths that contain custom themes here, relative to this directory. #html_theme_path = [] # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". #html_title = None # A shorter title for the navigation bar. Default is the same as html_title. #html_short_title = None # The name of an image file (relative to this directory) to place at the top # of the sidebar. #html_logo = None # The name of an image file (within the static path) to use as favicon of the # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 # pixels large. #html_favicon = None # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ['_static'] # Add any extra paths that contain custom files (such as robots.txt or # .htaccess) here, relative to this directory. These files are copied # directly to the root of the documentation. #html_extra_path = [] # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. #html_last_updated_fmt = '%b %d, %Y' # If true, SmartyPants will be used to convert quotes and dashes to # typographically correct entities. #html_use_smartypants = True # Custom sidebar templates, maps document names to template names. #html_sidebars = {} # Additional templates that should be rendered to pages, maps page names to # template names. #html_additional_pages = {} # If false, no module index is generated. #html_domain_indices = True # If false, no index is generated. #html_use_index = True # If true, the index is split into individual pages for each letter. #html_split_index = False # If true, links to the reST sources are added to the pages. #html_show_sourcelink = True # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. #html_show_sphinx = True # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. #html_show_copyright = True # If true, an OpenSearch description file will be output, and all pages will # contain a tag referring to it. The value of this option must be the # base URL from which the finished HTML is served. #html_use_opensearch = '' # This is the file name suffix for HTML files (e.g. ".xhtml"). #html_file_suffix = None # Output file base name for HTML help builder. htmlhelp_basename = 'shodan-pythondoc' # -- Options for LaTeX output --------------------------------------------- latex_elements = { # The paper size ('letterpaper' or 'a4paper'). #'papersize': 'letterpaper', # The font size ('10pt', '11pt' or '12pt'). #'pointsize': '10pt', # Additional stuff for the LaTeX preamble. #'preamble': '', } # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ ('index', 'shodan-python.tex', u'shodan-python Documentation', u'achillean', 'manual'), ] # The name of an image file (relative to this directory) to place at the top of # the title page. #latex_logo = None # For "manual" documents, if this is true, then toplevel headings are parts, # not chapters. #latex_use_parts = False # If true, show page references after internal links. #latex_show_pagerefs = False # If true, show URL addresses after external links. #latex_show_urls = False # Documents to append as an appendix to all manuals. #latex_appendices = [] # If false, no module index is generated. #latex_domain_indices = True # -- Options for manual page output --------------------------------------- # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ ('index', 'shodan-python', u'shodan-python Documentation', [u'achillean'], 1) ] # If true, show URL addresses after external links. #man_show_urls = False # -- Options for Texinfo output ------------------------------------------- # Grouping the document tree into Texinfo files. List of tuples # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ ('index', 'shodan-python', u'shodan-python Documentation', u'achillean', 'shodan-python', 'One line description of project.', 'Miscellaneous'), ] # Documents to append as an appendix to all manuals. #texinfo_appendices = [] # If false, no module index is generated. #texinfo_domain_indices = True # How to display URL addresses: 'footnote', 'no', or 'inline'. #texinfo_show_urls = 'footnote' # If true, do not generate a @detailmenu in the "Top" node's menu. #texinfo_no_detailmenu = False shodan-1.26.1/docs/tutorial.rst0000664000175000017500000000612413634247043020277 0ustar achilleanachillean00000000000000 Getting Started =============== Installation ------------------ To get started with the Python library for Shodan, first make sure that you've `received your API key `_. Once that's done, install the library via the cheeseshop using: .. code-block:: bash $ easy_install shodan Or if you already have it installed and want to upgrade to the latest version: .. code-block:: bash $ easy_install -U shodan It's always safe to update your library as backwards-compatibility is preserved. Usually a new version of the library simply means there are new methods/ features available. Connect to the API ------------------ The first thing we need to do in our code is to initialize the API object: .. code-block:: python import shodan SHODAN_API_KEY = "insert your API key here" api = shodan.Shodan(SHODAN_API_KEY) Searching Shodan ---------------- Now that we have our API object all good to go, we're ready to perform a search: .. code-block:: python # Wrap the request in a try/ except block to catch errors try: # Search Shodan results = api.search('apache') # Show the results print('Results found: {}'.format(results['total'])) for result in results['matches']: print('IP: {}'.format(result['ip_str'])) print(result['data']) print('') except shodan.APIError as e: print('Error: {}'.format(e)) Stepping through the code, we first call the :py:func:`Shodan.search` method on the `api` object which returns a dictionary of result information. We then print how many results were found in total, and finally loop through the returned matches and print their IP and banner. Each page of search results contains up to 100 results. There's a lot more information that gets returned by the function. See below for a shortened example dictionary that :py:func:`Shodan.search` returns: .. code-block:: python { 'total': 8669969, 'matches': [ { 'data': 'HTTP/1.0 200 OK\r\nDate: Mon, 08 Nov 2010 05:09:59 GMT\r\nSer...', 'hostnames': ['pl4t1n.de'], 'ip': 3579573318, 'ip_str': '89.110.147.239', 'os': 'FreeBSD 4.4', 'port': 80, 'timestamp': '2014-01-15T05:49:56.283713' }, ... ] } Please visit the `REST API documentation `_ for the complete list of properties that the methods can return. It's also good practice to wrap all API requests in a try/ except clause, since any error will raise an exception. But for simplicity's sake, I will leave that part out from now on. Looking up a host ----------------- To see what Shodan has available on a specific IP we can use the :py:func:`Shodan.host` function: .. code-block:: python # Lookup the host host = api.host('217.140.75.46') # Print general info print(""" IP: {} Organization: {} Operating System: {} """.format(host['ip_str'], host.get('org', 'n/a'), host.get('os', 'n/a'))) # Print all banners for item in host['data']: print(""" Port: {} Banner: {} """.format(item['port'], item['data'])) shodan-1.26.1/MANIFEST.in0000664000175000017500000000016713335617250016510 0ustar achilleanachillean00000000000000include AUTHORS include LICENSE include requirements.txt include CHANGELOG.md graft docs recursive-include shodan *.py shodan-1.26.1/LICENSE0000664000175000017500000000242313304067234015751 0ustar achilleanachillean00000000000000Copyright (c) 2014- John Matherly Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. Except as contained in this notice, the name(s) of the above copyright holders shall not be used in advertising or otherwise to promote the sale, use or other dealings in this Software without prior written authorization. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.