All Questions
Tagged with compression python
42 questions
2
votes
2
answers
197
views
Checking file header for magic number in Python
I wrote this to check files for successful compression with LZ4.
Any advice is welcome.
Particularly regarding raising exceptions and error handling.
...
- 767
1
vote
1
answer
129
views
Looking for help simplifying a general purpose compression function
The following is a general purpose file compression function with the following features:
It can take either a single file or directory as the source input
It defaults to the source directory when to ...
4
votes
1
answer
325
views
Python script for zlib decompression
I'm looking for feedback/improvement on code that I wrote. It's a function that scans a file for zlib headers and returns the header, offset, and decompressed data. In main, below the function, I use ...
- 41
3
votes
2
answers
263
views
Slow Huffman Code in pure Python
I was working on writing a fast implementation of a simple Huffman code compression of text. The idea was to write it using only the standard library, but I can't seem to find a way to make it faster. ...
- 141
4
votes
1
answer
378
views
Huffman encoding as Unix style command line utility
After watching Tom Scott explain Huffman coding in this YouTube video, I wanted to implement it myself. I want to use this project to further my understanding of Python. Additionally this tool should ...
- 143
3
votes
0
answers
217
views
Doing image compression with Neural Network AutoEncoders
I wanted to create an image compressor using Machine Learning and started work on an "AutoEncoder". This is a type of Neural Network which takes in the image and creates a compressed vector form.
It ...
- 151
6
votes
2
answers
336
views
Compressing large jpeg images
I'm working with thousands of large image files in a regularly updated library. The following script does the job (on average reduces my file size ~95%) but costs me around 25 seconds to compress one ...
- 93
8
votes
1
answer
499
views
Brainfuck compressor in Python
Just for fun, I created a script called TinyBF which can compress Brainfuck programs into a format half their size. For example, take a famous hello world program clocking in at 78 bytes:
...
- 445
3
votes
1
answer
95
views
File compressor
I have written a file compressor in python. The program generates a file filled with sequences of the same character with a random length, and compresses that file. I would like feedback on code ...
- 10.7k
2
votes
0
answers
110
views
Google drive backup robot
I have a lot of files. I care about those files. I also care about good programming practices and learning proper python. Given the described context, it seemed appropriate to implement my own ...
- 307
3
votes
0
answers
105
views
Variable bit length lossy floating point compression
I am implementing a new compression algorithm for the weights of a neural network for the Leela Chess project. the weights are roughly 100Mb of ...
- 3,707
2
votes
2
answers
2k
views
Reading a gzipped file with JSON data for thousands of users
I'm a newbie in Python.
I need to read a GZ file contains thousands of users.
But I think my code is not optimized in this case. Could anyone can help me? What should I do to improve performance? Or ...
- 145
2
votes
1
answer
8k
views
Read gzipped JSON file from URL
I am requesting a JSON file that's gzipped. First, I download file:
...
- 145
8
votes
2
answers
4k
views
Append to compressed tar file with performance
As you might know, Python's tarfile does not have the ability to append to the files that compressed with e.g. gz or ...
- 185
4
votes
2
answers
96
views
Bernoulli Sequence Lab
I'm writing some Python 2 code with which to analyze the compressibility of random bitstrings. It's working pretty well right now, and now I'd like to request some help making sure it follows PEP 8 ...
- 141