Shrinking (compressing) and restoring (decompressing) of data.
1
vote
0answers
18 views
xzcmp does not work for some files. (Internal pipe error 141?)
I am using xzcmp to compare several XZ files. But I have a strange problem: For some files (*), xzcmp doesn't work and returns exit code 2, without error message.
zxcmp version1.sql.xz version2.sql....
-2
votes
2answers
53 views
7zip, xz, gzip, tar, etc — what should I use? [closed]
What would you recommend me to use: 7zip, xz, gzip, tar, etc which is more or less available on the most linux distributives and what does compression the best?
1
vote
1answer
23 views
Is there any way to check the compression ratio of a xz compressed file?
In windows by default one can see the compression ratio of zip files and after installing WinRAR it also supports this feature but currently I am using a linux and have compressed a file using lzma2 ...
0
votes
1answer
18 views
Is there any way to verify the integrity of a xz archive without decompressing the entire archive?
As the title says it all can I check the integrity of the archive without decompressing it as the archive is 64GB. It is a backup of my old laptop and before i format my old laptop I wanted to verify ...
0
votes
1answer
32 views
A generic command to extract archive files [duplicate]
There are many archive formats: tar, tar.gz, tar.bz2, zip, rar, tar.xz and more. I would like to have a single command that would determine the type of archive I give to it as input and extract the ...
1
vote
1answer
26 views
Is safe to use rsync -z option for database sync (Postgresql binary replication)?
I see that this option is good for slow connections, because it enables the compression. But what this really means?
Will this option compress the data only during the transfer?
Or the files in the ...
0
votes
2answers
58 views
Compress ls result using tar
I want to compress log files by specific months.
Listing all files with following commands
ll | grep 'Sep' | grep 'oim_server'
Then I need to compress with tar result of it
How can I do it?
19
votes
2answers
2k views
Zipped hard drive image very big
I just made an image of a freshly installed dual boot (Ubuntu and Windows) using this command (which I've been using for a while for smaller images):
dd if=/dev/sda | gzip > /mnt/drive.img.gz
On ...
2
votes
3answers
156 views
Tool to compress/decompress STDIN
Is there a tool that compresses STDIN, and outputs to STDOUT? This tool (or its counterpart) should be able to decompress as well. Something simple is fine, but it must be lossless.
3
votes
1answer
94 views
How to load compressed kernel modules in Ubuntu?
I would like to use LZMA-compressed kernel modules on my system. Unfortunately Canoncial leaves that feature disabled both in kernel and user-space tools. Here's what I did so far:
Compile and ...
2
votes
1answer
45 views
Putting m files in a directory into n tar files by size?
Does anyone know if it's possible to create n tar files (of roughly equal size) out of a larger collection of files in a directory, in such a way that they can be extracted individually?
I was ...
1
vote
1answer
25 views
Get compressed size of compressed file with redirect to /dev/null?
I have a compressed file, and on compression or decompression, I want to get it's final compressed size (for compression) or original compressed size (for decompression).
I have used wc -c before:
$...
1
vote
0answers
26 views
Enable cuda support in libbsc
I am interested in using libbsc in CUDA support. libbsc is basically used for compression in GPU (as well as CPU). As far as I tested, it is running in CPU (openmp) not in NVIDIA GPU (CUDA driver ...
0
votes
1answer
53 views
Does Redhat and CentOS come with zlib preinstalled? [closed]
Does Redhat and CentOS come with zlib preinstalled?
If yes, which versions have it?
// need to know if I can expect users to have it on their Servers, since I can't install it there, and can't ask ...
3
votes
3answers
261 views
why should I use tar.xz instead of tar.gz? xz is a lossless data compression program and file format [duplicate]
Today first time in my life I saw tar.xz download. I searched the internet and found Wikipedia articles (xz and XZ Utils)
Interesting quote about the users of xz
xz has gained notability for ...
1
vote
0answers
149 views
Proper way to create a compressed, rsyncable mirror of a directory?
I would like to create a compressed mirror of some directories in such a way that it can be rsynced again, both from the original destination and to a remote destination.
I can do it inefficiently in ...
0
votes
1answer
32 views
Reduce a size of a file(.jar) in unix
I want to reduce the big file (*.jar) in unix.
i have tried with tar,bzip,gzip and zip commands. but all are same result not compressed much.
Could any one help on this ???
0
votes
0answers
67 views
How to deal with corrupted .tar.lrz file
I have a file file.tar.lrz and tried to unzip it via lrzuntar file.tar.lrz but it gives an error: "Not an lrzip file" with lrunzip.
I also tried to open it with file roller. file file.tar.lrz says ...
2
votes
1answer
210 views
How can I decompress and print the last few lines of a compressed text file?
I have 6 gzipped text files, each of which is ~17G when compressed. I need to see the last few lines (decompressed) of each file to check whether a particular problem is there. The obvious approach is ...
1
vote
1answer
87 views
How to compress all files in folder and erase the untared versions via command line?
I have a backup disk containing uncompressed versions of my files that I would like to compress to save space on this disk. The disk (containing files to compress) is full (about 3TB with 17GB free). ...
2
votes
3answers
128 views
Work with compressed dd images
How can I compress dd images and work with them at the same time?
On Windows, I can mark it as compressed and work with it like a normal file.
I can't re-partition, but I can mount some drives (For ...
1
vote
1answer
127 views
On the fly compression of read- and writeable folder (like Windows EFS)?
Is there a possibility to compress the contents of a directory in Unix (especially AIX) on the fly, while still being able to read and write in the directory? Like the Windows EFS encryption does?
4
votes
2answers
121 views
Print archive file list instantly (without decompressing entire archive)
A problem with .tar.gz archives is that, when I try to just list an archive's content, the computer actually decompresses it, which would take a very long time if the file is large.
Other file ...
1
vote
1answer
91 views
Compressing a folder but do not compress specific file types but include them in the gz file
I am using tar with pigz to compress a folder and save a backup. Size of this folder ~250 GB or more. This folder has variety of content including numerous text and log files, ISOs and zip files in ...
3
votes
2answers
60 views
tar while avoiding compression of certain file types
I'm trying to create a very large backup of an entire hard disk, using tar. However, there are many files such as CR2 (Canon RAW 2), JPG and a few other files which are already compressed.
I have ...
3
votes
1answer
105 views
Corrupted gz file
I have a MySQL database backup in a gz file. When trying to uncompress it I get the following:
gzip: db_stepup.sql.gz: not in gzip format
I read that sometimes is just a matter of removing the gz ...
0
votes
2answers
65 views
NTFS compression and RSync
I currently have a hard drive that I backup using RSync. I do weekly; monthly snapshots. I would like to compress some of my hard drive using NTFS compression. How would that affect my future ...
1
vote
0answers
35 views
Where is my 7za data being written to
I have a weird question about 7za on Centos 7. I am adding a file to a 7za archive using the following command (the total size of the directory is around 40GB)
/usr/bin/7za a -ppassword -t7z -mmt=off ...
0
votes
3answers
307 views
Compress and remove original directory with limited storage
On my 3 TB external hard drive, I have a 2.7 TB directory containing relatively small files. I would like to compress this 2.7 TB directory and remove it to keep only the compressed version. The issue ...
0
votes
1answer
41 views
how to compress all folders in a directory
I have a directory full of folders and I was wondering
if there is any command to compress all the folders in a directory X with date in the name of the file and not update old ones in case they ...
94
votes
8answers
13k views
Why would I tar a single file?
At my company, we download a local development database snapshot as a db.dump.tar.gz file. The compression makes sense, but the tarball only contains a single file (db.dump).
Is there any point to ...
0
votes
1answer
79 views
gzip - is there a risk to lose files during compression if process dead?
Is there a risk of losing files during compression if process dies? I'm gzipping big files using wildcard, but accidentally running it without screen or nohup. Is there a risk to lose source files if ...
3
votes
1answer
118 views
GNU sort --compress-program compressing only first temporary
I am sorting big files (>100Go), and to reduce time spent on disk writes, I am trying to use GNU sort's --compress-program parameter. (Related: How to sort big files?)
However, it appears in certain ...
1
vote
1answer
203 views
How do I reduce the size of a pdf file that contains images?
I have a pdf file that contains images and I want to reduce its size in order to upload it to a site with a size limit.
So, how can I reduce the size of a pdf file from the command-line?
0
votes
1answer
125 views
Linux Gzip increasing size
I try to gzip a file abc.log which has size of 111 bytes, but after gzip, the size of the file increased to 125 bytes, why is that? Is it when i perform gzip, it will create header and trailer that ...
2
votes
2answers
235 views
Linux + how to compress the old log files automatically
Under /var/log/httpd folder, files are created with huge capacity!!!
What solution can I use in order to compress the old files automatically (can logrotate handle this)?
If yes, how to configure ...
2
votes
3answers
158 views
Unable to use tar to run an archive through bzip2 compression
I am having difficulty compression an archive with bzip using tar.
The archive I have is user-logs.tar.
I am attempting to run this:
tar -jf user-logs.tar
I get this error:
tar: You must ...
1
vote
1answer
87 views
linux unbuffered compression options
I'm working on a embedded system and due to persistent memory restrictions, I need to compress a log file "on the fly".
My objective is to have a one liner inside a script, which would look like this:...
1
vote
2answers
56 views
Squashfs minimal compression?
I'd like to create a squashfs image of a directory, but it's a huge one and even minimal compression would be a big win. Is there a way to do something like mksquashfs -1?
3
votes
1answer
129 views
Is there a compression method that supports solid compression and also adding data to the compressed file?
I have a large compressed .tar.xz file containing log files. The compression ratio is very good - but it takes a long time to compress, and if I want to add additional log files to it, I have to ...
1
vote
0answers
74 views
On-the-fly decompression
I have 44 TB worth of gzipped files that are 1.5 GB each when compressed or 2.0 GB when uncompressed. I have a tool that can read only uncompressed files. I would like to avoid the overhead of ...
1
vote
1answer
400 views
log compression using logrotate
I was trying to set compression for jboss log files. The log files I want to compress are console.log and server.log. Compression for console.log is working fine, but server.log I am seeing issues. I ...
0
votes
2answers
142 views
Tar big files in different directories
I have ~20 files, each about 10 GB in size, in different folders that I need to compress into a single tar. It is not practical to copy them all to one location. More precisely, the files are ...
0
votes
1answer
355 views
Compressing files using p7zip from the command-line
I would like to compress four ISO files using 7z into a new archive called ISOs.7z from the command-line on my Sabayon machine. These are the commands I have tried so far (I know none of these specify ...
0
votes
1answer
38 views
Verify files AFTER extracting gzip
I gzip a folder, transfer it to a remote location and then extract it to a folder. The problem is that this machine is a little unreliable and sometimes writes don't succeed and there can be zero-byte ...
3
votes
2answers
178 views
Extract compressed file by it's header as it is piped from stdout (locally or from remote location)?
I am sending compressed file with piping either local or from a network location. And on the receiving end, I would like to detect the type of compression and use the appropriate decompression utility ...
1
vote
1answer
515 views
How to convert tar.bz2 to .bz2?
I have a file with compression tar.bz2 , I would like to convert it to .bz2 . Is there a command that does this in one shot?
I tried the below but no joy !
tar -xvjf US.tar.bz2
US
bzip2: ...
2
votes
3answers
307 views
How can I keep rsync from using encryption and compression for the file transfers after the connection is authorized?
I understand that a secure connection (i.e. ssh) is needed to authorize a connection to the remote server. But after that is authorized can the data be transmitted without encryption and compression ...
5
votes
1answer
334 views
Get compressed size of piped output with gzip?
To get uncompressed size of already compressed file, I can use -l option in gzip utility:
gzip -l compressedfile.gz
However is there a way to get size of compressed file if I am piping the output? ...
22
votes
4answers
2k views
How to know how much space an uncompressed zip will take
Given a (really long) list of zip files, how can you tell the size of them once uncompressed?
