Tell me more ×
Stack Overflow is a question and answer site for professional and enthusiast programmers. It's 100% free, no registration required.

Is there an easy way to run a MySQL query from the linux command line and output the results in csv format?

Here's what I'm doing now:

mysql -u uid -ppwd -D dbname << EOQ | sed -e 's/        /,/g' | tee list.csv
select id, concat("\"",name,"\"") as name
from students
EOQ

It gets messy when there are a lot of columns that need to be surrounded by quotes, or if there are quotes in the results that need to be escaped.

share|improve this question
You can use REPLACE() in your query to have the quotes escaped. – dsm Dec 10 '08 at 16:08
up vote 302 down vote accepted

From http://www.tech-recipes.com/rx/1475/save-mysql-query-results-into-a-text-or-csv-file/

SELECT order_id,product_name,qty
FROM orders
INTO OUTFILE '/tmp/orders.csv'
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
share|improve this answer
34  
Nice, but this is useless if the database is at remote server where I cannot create files (results in "Access denied for user..."). Any recipe? – Tomas Oct 22 '11 at 9:49
1  
@TomasT. there are a bunch of solutions below that get you tab delimited instead of CSV. Excel, for instance, can take both CSV and tab delimited, and convert one to the other. – Paul Tomblin Oct 22 '11 at 12:32
3  
@Tomas if you have access to a remote filesystem and MySQL, you must be able to write somewhere. instead of /tmp, try /home/yourusername/file.csv -- if that fails, and the result set is not that large, you could copy the output from your SSH client and paste to your local machine. – Michael Butler Mar 9 '12 at 15:40
17  
The question specified MySQL, not "standards compliant". – Paul Tomblin Jul 11 '12 at 13:28
6  
How to include header as well? – Bogdan Gusiev Jan 25 at 10:01
show 7 more comments
$ mysql your_database --password=foo < my_requests.sql > out.csv

Which is tab separated. Pipe it like that to get a true CSV (thanks @therefromhere):

... .sql | sed 's/\t/,/g' > out.csv
share|improve this answer
5  
This doesn't solve the need to output the results in csv. – FilmJ Apr 3 '11 at 21:34
3  
yes IT does.... – Stan May 4 '11 at 12:03
17  
It's tab-separated, not comma-separated. – Flimm Aug 30 '11 at 15:13
8  
@Flimm, assuming you don't have embedded commas/tabs in the fields you can convert it by piping the result into | sed 's/\t/,/g' – therefromhere Nov 10 '11 at 4:42
8  
the sed 'fix' does not compensate for commas that may appear in any of the selected data and will skew your columns outputted accordingly – Joey T Dec 11 '12 at 1:17
show 7 more comments

mysql --batch, -B

Print results using tab as the column separator, with each row on a new line. With this option, mysql does not use the history file. Batch mode results in non-tabular output format and escaping of special characters. Escaping may be disabled by using raw mode; see the description for the --raw option.

This will give you a tab separated file. Since commas (or strings containing comma) are not escaped it is not straightforward to change the delimiter to comma.

share|improve this answer
3  
this is a preferred solution: 1) tab-separated-value lists are common in UNIX 2) TSV imports are natively supported by most import systems, including Excel, OpenOffice Spreadsheets, etc. 3) no need to escape quote characters for text fields 4) makes command-line exports a breeze – Joey T Dec 11 '12 at 1:18
2  
this is the best solution because unlike first one need not have permissions on servers like RDS – muayyad alsadi Feb 13 at 10:31
1  
A neat trick: if you save the tab separated file as .xls instead of .csv, it will open in excel without any need for "text to data" conversion and without any regional settings issues. – serbaut Apr 11 at 21:05

Here's a fairly gnarly way of doing it. Found it somewhere, can't take any credit

mysql --user=wibble --password wobble -B -e "select * from vehicle_categories;" | sed "s/'/\'/;s/\t/\",\"/g;s/^/\"/;s/$/\"/;s/\n//g" > vehicle_categories.csv

Works pretty well. Once again though a regex proves write only.


Regex Explanation:

  • s/// means substitute what's between the first // with what's between the second //
  • the "g" at the end is a modifier that means "all instance, not just first"
  • ^ (in this context) means beginning of line
  • $ (in this context) means end of line

So, putting it all together:

s/'/\'/          replace ' with \'
s/\t/\",\"/g     replace all \t (tab) with ","
s/^/\"/          at the beginning of the line place a "
s/$/\"/          at the end of the line place a "
s/\n//g          replace all \n (newline) with nothing
share|improve this answer
The -e flag was exactly what I was looking for! Thanks! – Gaurav Gupta Jan 3 '12 at 8:46
This regex is quite simple really; like a lot of other answers on this page, its just cleaning up the output from mysql -B. If you separated the regex into individual statements on separate lines, it would be quite simple (for someone who knows regex) to understand. – David Mar 9 '12 at 0:58
Since comments can't have formatting, I'm going to edit the answer to explain the regex... – fiXedd May 3 '12 at 17:08
This is fantastic. Thanks! – Raj Oct 15 '12 at 18:55
1  
Great sed regex. Thanks. – Cerin Mar 12 at 16:21
show 1 more comment

The OUTFILE solution given by Paul Tomblin causes a file to be written on the MySQL server itself, so this will work only if you have FILE access, as well as login access or other means for retrieving the file from that box.

If you don't have such access, and tab-delimited output is a reasonable substitute for CSV (e.g., if your end goal is to import to Excel), then Serbaut's solution (using mysql --batch and optionally --raw) is the way to go.

share|improve this answer

How about:

mysql your_database -p < my_requests.sql | awk '{print $1","$2}' > out.csv
share|improve this answer
2  
I really like this one. It is much cleaner, and I like the use of awk. However, I would have probably gone with this: mysql -uUser -pPassword your_database < my_requests.sql | awk 'BEGIN{OFS="=";} {print $1,$2}' > out.csv – Josh Apr 11 '12 at 0:00
Pure command line solution and works perfectly – chawkinsuf Feb 20 at 22:50

Unix/cygwin only, pipe it through 'tr':

mysql <database> -e "<query here>" | tr '\t' ',' > data.csv

N.B.: this handles neither embedded commas, nor embedded tabs.

share|improve this answer

MySQL Workbench can export recordsets to CSV, and it seems to handle commas in fields very well. The CSV opens up in OpenOffice fine.

share|improve this answer
Thanks a million David. After spending 3 hours getting the newlines to output properly for HTML content in the data, I used the MySQL Workbench and in 2 minutes I had my CSV file ready. – mr-euro Jul 11 '12 at 17:02
I've just found it can save as XML, too, which is great. I'm hoping to migrate from one application to another by using XSLT to transform this XML into a CSV file suitable for importing into the target application. – David Oliver Aug 5 '12 at 21:27

Here's what I do:

echo $QUERY | \
  mysql -B  $MYSQL_OPTS | \
  perl -F"\t" -lane 'print join ",", map {s/"/""/g; /^[\d.]+$/ ? $_ : qq("$_")} @F ' | \
  mail -s 'report' person@address

The perl script (sniped from elsewhere) does a nice job of converting the tab spaced fields to CSV.

share|improve this answer

Alternatively to the answer above, you can have a MySQL table that uses the CSV engine.

Then you will have a file on your hard disk that will always be in a CSV format which you could just copy without processing it.

share|improve this answer

If there is a PHP installed on the machine you are using, you can write a PHP script to do that. It requires the PHP installation has the mysql extension installed.

You can call PHP interpreter from the command line like so

php --php-ini path/to/php.ini your-script.php

I am including the --php-ini switch, because you may need to use your own php config that enables the mysql extension. On PHP 5.3.0+ that extension is enabled by default, so that is no longer necessary to use the config to enable it.

Then you can write your export script like any normal PHP script.

<?php
#mysql_connect("localhost", "username", "password") or die(mysql_error());
mysql_select_db("mydb") or die(mysql_error());

$result = mysql_query("SELECT * FROM table_with_the_data p WHERE p.type = $typeiwant");

$result || die(mysql_error());

while($row = mysql_fetch_row($result)) {
  $comma = false;
  foreach ($row as $item) {
    #make it comma separated
    if ($comma) {
      echo ',';
    } else {
      $comma = true;
    }
    #quiote the quiotes
    $quoted = str_replace("\"", "\"\"", $item);

    #quiote the string
    echo "\"$quoted\"";
  }
    echo "\n";
}
?>

The advantage of this method is, that it has no problems with varchar and text fields, that have text containing newlines. Those fields are correctly quioted and those newlines in them will be interpreted by the CSV reader as a part of the text, not record separators. That is something that is hard to correct afterwards with sed or so.

share|improve this answer
This is unnecesary complicated, indeed. – Fran Aug 12 '12 at 17:13

Using the solution posted by Tim, I created this bash script to facilitate the process (root password is requested, but you can modify the script easily to ask for any other user):

#!/bin/bash

if [ "$1" == "" ];then
    echo "Usage: $0 DATABASE TABLE [MYSQL EXTRA COMMANDS]"
    exit
fi

DBNAME=$1
TABLE=$2
FNAME=$1.$2.csv
MCOMM=$3

echo "MySQL password:"
stty -echo
read PASS
stty echo

mysql -uroot -p$PASS $MCOMM $DBNAME -B -e "SELECT * FROM $TABLE;" | sed "s/'/\'/;s/\t/\",\"/g;s/^/\"/;s/$/\"/;s/\n//g" > $FNAME

It will create a file named: database.table.csv

share|improve this answer

I use SQLyog for this. You can download the free version from here.

share|improve this answer

This is simple and it works on anything without needing batch mode or output files.

select concat_ws(',', 
    concat('"',replace(field1,'"','""'),'"'), 
    concat('"',replace(field2,'"','""'),'"'), 
    concat('"',replace(field3,'"','""'),'"')) 

from your_table where etc;

Explanation:

  1. Replace " with "" in each field --> replace(field1,'"','""')
  2. Surround each result in quotation marks --> concat('"',result1,'"')
  3. Place a comma between each quoted result --> concat_ws(',', quoted1, quoted2, ...)

That's it!

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.