current community

your communities

more stack exchange communities

Stack Exchange
tour help
Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.
up vote 0 down vote favorite
1

I want to create a service where people can upload files. However, since file storage costs money, I want to compress the files so they take less space. I would want to write my own compression algorithm, however, PHP doesn't have good ways to handle binary data (which is needed for many compression algorithms). So I wondered, what would be a better language to create such a website in?

I have knowledge of PHP (and Javascript, HTML and CSS) but no experience with other things like Ruby, Perl, Python, and other web development languages.

share|improve this question

closed as off-topic by gnat, Bart van Ingen Schenau, Kilian Foth, MichaelT, Dan Pichelman Dec 17 '14 at 21:23

This question appears to be off-topic. The users who voted to close gave these specific reasons:

  • "Questions about what language, technology, or project one should take up next are off topic on Programmers, as they can only attract subjective opinions for answers. There are too many individual factors behind the question to create answers that will have lasting value. You may be able to get help in The Whiteboard, our chat room." – Bart van Ingen Schenau, Dan Pichelman
  • "Questions asking us to recommend a tool, library or favorite off-site resource are off-topic for Programmers as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it." – gnat, Kilian Foth, MichaelT
If this question can be reworded to fit the rules in the help center, please edit the question.
8  
Compression algorithms are a seriously researched area. Do you really think you are going to build a better one than the dozen or so out there? –  Matthew Flynn Nov 27 '11 at 22:45
2  
No, I don't. However, I would love to try writing one myself. :) –  Qqwy Nov 27 '11 at 22:47
1  
C/C++ is pretty awesome for this –  Raynos Nov 27 '11 at 23:25
2  
Designing and implementing a compression algorithm is a very different project from a storage service. The best way would be to write your compressor in C and your service app in whatever you know best (PHP in your case). That way you use the best tool for each one, and the success/failure of one isn't tied to the other. –  Javier Nov 28 '11 at 3:53
3  
BTW, keep in mind that the vast majority of space-hogging user files (images, music, video) are already compressed, so you'd be hard pressed to improve on that by even a few percent. Specially since most of these are content-specific lossy algorithms. IMHO, adding compression to the service would only be worth if your target demographic suggest lots of non-compressed data or huge opportunities for deduplication (for example, you're likely to get lots of direct copies from a single popular DVD movie. but if theyr'e recompressed, forget about it.) –  Javier Nov 28 '11 at 3:58

4 Answers 4

up vote 6 down vote accepted

PHP supports a variety of compression algorithms and popular compression libraries, like Bzip2, LZF and Rar, and for most common scenarios it provides excellent support for working with binary data. There's at least one working PHP implementation of a compression algorithm out there, proving that it's possible. PHP and any other interpreted language may prove inefficient for the task, but efficiency issues highly depend on the specifics of the implementation and the actual usage of your library, for which you don't provide enough information.

I would advise you to not try and create your own compression algorithm and use an established library instead, there's no point in reinventing the wheel. You don't have to use PHP if you don't want to, every web oriented language out there has support for popular compression libraries. You can use perl's Gzip and Zlib core modules, or Python's gzip or bzip2 modules. I don't know what kind of support for compression Ruby has, but this StackOverflow question can help you, if you choose Ruby.

But if for any reason you really want to write your own original compression algorithm, you should probably choose C. C is extremely portable* and if you manage to actually built a compression library then you can use it from PHP via an extension or in Python via a module. Both PHP extensions and Python modules are also commonly written in C, and it would be fairly easy to write one of those if you coped with writing a compression algorithm. And there's always the old school option of C web programming via CGI, if you don't mind diving into a world of hurt.

* There are C compilers for almost every imaginable platform and ways to use C libraries from almost every other language.

share|improve this answer
1  
+1 for no wheel-reinvention. Compression is a VERY well studied field and whatever you might roll of your own just isn't going to perform as well as something you'll get from somebody else. Plus it's probably free and already has bindings to your language of choice. –  Dan Ray Nov 28 '11 at 13:17
up vote 1 down vote

Haskell, Ruby, Python, Java and other JVM languages, and C# and other .NET languages all have good support for both web programming and programming against binary data.

I personally would find Ruby and Python to be the simplest to get into.

share|improve this answer
up vote 1 down vote

Any language that allows you to shuffle bits and bytes can be used to write a compressor. I'm pretty sure PHP lets you shuffle bits and bytes but it's been a while since I wrote anything in it. Various javascript environments like Node.js also have support for binary data so it's not a matter of which web-oriented language would work best with binary data but how much of a performance penalty you're willing to take by writing a compressor in one.

share|improve this answer
up vote 1 down vote

In truth you best bet it to use a pre existing compression package. Really you are not going to do better than gzip or bzip2 or the like.

Also remember if they are uploading images, sound or video then the files are already compressed so you will not be able to compress them farther.

That being said if you want to work with Binary data check out Erlang. It is an amazing langauge and it has really cool syntax for working with bits.

Disclaimer: I wrote a book on Erlang

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.