/sqlitebiter

A CLI tool to convert CSV/Excel/HTML/JSON/LTSV/Markdown/TSV/Google-Sheets to a SQLite database file.
  1. Python 98.4%
  2. Batchfile 1.6%
Python Batchfile

Clone with HTTPS

Use Git or checkout with SVN using the web URL.

Download ZIP
Find file
Switch branches/tags
Nothing to show
Latest commit cc9ebb7 Feb 26, 2017 @thombashi committed on GitHub Merge pull request #30 from thombashi/develop 
Bump version to 0.6.2
Permalink
Failed to load latest commit information.
docs Update README Feb 26, 2017
requirements Update requirements Feb 26, 2017
sqlitebiter Bump version to 0.6.2 Feb 26, 2017
test Fix test cases to update dependency packages. Jan 8, 2017
.gitignore Update .gitignore Jun 19, 2016
.travis.yml Add automated package deploy Feb 26, 2017
LICENSE Initial commit May 4, 2016
MANIFEST.in Update MANIFEST.in Oct 22, 2016
README.rst Update README Feb 26, 2017
appveyor.yml Add Python 3.6 support Feb 26, 2017
cli.py Initial commit Jan 1, 2017
packaging.bat Update packaging Jan 9, 2017
setup.cfg initial commit May 4, 2016
setup.py Bump version to 0.6.2 Feb 26, 2017
tox.ini Add Python 3.6 support Feb 26, 2017

README.rst

sqlitebiter

Linux CI test status Windows CI test status GitHub repository

Summary

A CLI tool to convert CSV/Excel/HTML/JSON/LTSV/Markdown/TSV/Google-Sheets to a SQLite database file.

Features

Usage

Create SQLite database from files

docs/gif/usage_example.gif

Create SQLite database from URL

Following is an example that convert HTML table tags within a web page to SQLite tables.

$ sqlitebiter -v url "https://en.wikipedia.org/wiki/Comparison_of_firewalls"
[INFO] convert 'https://en.wikipedia.org/wiki/Comparison_of_firewalls' to 'Comparison_of_firewalls_Wikipedia_html1 (Firewall TEXT, License TEXT, CostUsageLimits TEXT, OS TEXT)' table
[INFO] convert 'https://en.wikipedia.org/wiki/Comparison_of_firewalls' to 'Comparison_of_firewalls_Wikipedia_html2 (Firewall TEXT, License TEXT, Cost TEXT, OS TEXT)' table
[INFO] convert 'https://en.wikipedia.org/wiki/Comparison_of_firewalls' to 'Comparison_of_firewalls_Wikipedia_html3 (CanTarget TEXT, Changingdefaultpolicytoacceptrejectbyissuingasinglerule TEXT, IPdestinationaddresses TEXT, IPsourceaddresses TEXT, TCPUDPdestinationports TEXT, TCPUDPsourceports TEXT, EthernetMACdestinationaddress TEXT, EthernetMACsourceaddress TEXT, Inboundfirewallingress TEXT, Outboundfirewallegress TEXT)' table
[INFO] convert 'https://en.wikipedia.org/wiki/Comparison_of_firewalls' to 'Comparison_of_firewalls_Wikipedia_html4 (Can TEXT, [workatOSILayer4statefulfirewall] TEXT, [workatOSILayer7applicationinspection] TEXT, ChangeTTLTransparenttotraceroute TEXT, ConfigureREJECTwithanswer TEXT, DMZdemilitarizedzoneallowsforsingleseveralhostsnottobefirewalled TEXT, Filteraccordingtotimeofday TEXT, RedirectTCPUDPportsportforwarding TEXT, RedirectIPaddressesforwarding TEXT, FilteraccordingtoUserAuthorization TEXT, TrafficratelimitQoS TEXT, Tarpit TEXT, Log TEXT)' table
[INFO] convert 'https://en.wikipedia.org/wiki/Comparison_of_firewalls' to 'Comparison_of_firewalls_Wikipedia_html5 (Features TEXT, ConfigurationGUItextorbothmodes TEXT, [RemoteAccessWebHTTPTelnetSSHRDPSerialCOMRS232] TEXT, Changeruleswithoutrequiringrestart TEXT, Abilitytocentrallymanageallfirewallstogether TEXT)' table
[INFO] convert 'https://en.wikipedia.org/wiki/Comparison_of_firewalls' to 'Comparison_of_firewalls_Wikipedia_html6 (Features TEXT, Modularitysupportsthirdpartymodulestoextendfunctionality TEXT, [IPS : Intrusion prevention system] TEXT, OpenSourceLicense TEXT, [supports IPv6 ?] TEXT, ClassHomeProfessional TEXT, OperatingSystemsonwhichitruns TEXT)' table
[INFO] convert 'https://en.wikipedia.org/wiki/Comparison_of_firewalls' to 'Comparison_of_firewalls_Wikipedia_html7 (Can TEXT, [NAT44staticdynamicwoportsPAT] TEXT, [NAT64NPTv6] TEXT, IDSIntrusionDetectionSystem TEXT, VPNVirtualPrivateNetwork TEXT, AVAntiVirus TEXT, Sniffer TEXT, Profileselection TEXT)' table

For more information

More examples are available at http://sqlitebiter.rtfd.io/en/latest/pages/usage/index.html

Installation

Install via pip (recommended)

sqlitebiter can be installed via pip (Python package manager).

sudo pip install sqlitebiter

Python package dependencies are as follows.

Mandatory Python package dependencies

Mandatory Python packages are automatically installed during sqlitebiter installation via pip.

Google Sheets dependencies

Manual installation required to use Google Sheets feature.

Test dependencies

Misc

  • lxml (Faster HTML convert if installed)

Installing executable file in Windows

  1. Navigate to https://github.com/thombashi/sqlitebiter/releases
  2. Download the latest version of the sqlitebiter_win_x64.zip
  3. Unzip the file

Documentation

http://sqlitebiter.rtfd.io/