current community

your communities

more stack exchange communities

Stack Exchange
tour help
Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.
up vote 53 down vote favorite
14

I'm running pdftoppm to convert a user-provided PDF into a 300DPI image. This works great, except if the user provides an PDF with a very large page size. pdftoppm will allocate enough memory to hold a 300DPI image of that size in memory, which for a 100 inch square page is 100*300 * 100*300 * 4 bytes per pixel = 3.5GB. A malicious user could just give me a silly-large PDF and cause all kinds of problems.

So what I'd like to do is put some kind of hard limit on memory usage for a child process I'm about to run--just have the process die if it tries to allocate more than, say, 500MB of memory. Is that possible?

I don't think ulimit can be used for this, but is there a one-process equivalent?

share|improve this question

migrated from stackoverflow.com Aug 8 '12 at 1:45

This question came from our site for professional and enthusiast programmers.

5 Answers 5

up vote 19 down vote accepted

There's some problems with ulimit. Here's a useful read on the topic: Limiting time and memory consumption of a program in Linux, which lead to the timeout tool, which lets you cage a process (and it's forks) by time or memory consumption.

share|improve this answer
up vote 0 down vote

In addition to the tools from daemontools suggested by Mark Johnson, you can also consider chpst whish is found in runit. Runit itself is bundeled in busybox hence, you might already have it installed.

The man page of chpst shows the option:

-m bytes limit memory. Limit the data segment, stack segment, locked physical pages, and total of all segment per process to bytes bytes each.

share|improve this answer
up vote 12 down vote

Another way to limit this is to use Linux's control groups. This is especially useful if you want to limit a process's (or group of processes') allocation of physical memory distinctly from virtual memory. For example:

$ cgcreate -g memory:/myGroup
$ echo $(( 500 * 1024 * 1024 )) > /sys/fs/cgroup/memory/myGroup/memory.limit_in_bytes
$ echo $(( 5000 * 1024 * 1024 )) > /sys/fs/cgroup/memory/myGroup/memory.memsw.limit_in_bytes

will create a control group named "myGroup", cap the set of processes run under myGroup up to 500 MB of physical memory and up to 5000 MB of swap. To run a process under the control group:

$ cgexec -g memory:myGroup pdftoppm

Note that on a modern Ubuntu distribution this example requires installing the cgroup-bin package and editing /etc/default/grub to change GRUB_CMDLINE_LINUX_DEFAULT to:

GRUB_CMDLINE_LINUX_DEFAULT="cgroup_enable=memory swapaccount=1"

and then rebooting to boot with the new kernel boot parameters.

share|improve this answer
    
this is gold.. thanks for the answer! –  Dominik Dorn Mar 16 at 22:40
up vote 26 down vote

If your process doesn't spawn more children that consume the most memory, you may use setrlimit function. More common user interface for that is using ulimit command of a shell:

$ ulimit -Sv 500000     # Set ~500 mb limit
$ pdftoppm ...

This will only limit "virtual" memory of your process, taking into account—and limiting—the memory the process being invoked shares with other processes, and the memory mapped but not reserved (Java's large heap, for instance). Still, virtual memory is the closest approximation for processes that grow really large, making the said errors insignificant.

If your program spawns children, and it's they which allocate memory, it becomes more complex, and you should write axillary scripts to run processes under your control. I wrote in my blog, why and how.

share|improve this answer
    
why is setrlimit more complex for more children? man setrlimit tells me that "A child process created via fork(2) inherits its parents resource limits. Resource limits are preserved across execve(2)" –  akira Feb 13 '11 at 8:13
3  
Because the kernel does not sum the vm size for all child processes; if it did it would get the answer wrong anyway. The limit is per-process, and is virtual address space, not memory usage. Memory usage is harder to measure. –  MarkR Feb 13 '11 at 8:17
1  
if i understand the question correctly then OP whats the limit per subprocess (child) .. not in total. –  akira Feb 13 '11 at 8:21
    
@MarkR, anyway, virtual address space is a good approximation for the memory used, especially if you run a program that's not controlled by a virtual machine (say, Java). At least I don't know any better metric. –  Pavel Shved Feb 13 '11 at 8:23
    
Virtual address space is the best we really have; there isn't an easily measurable alternative. The pages measured don't need to be in core, they don't need to be private to that process, they're still counted. –  MarkR Feb 13 '11 at 8:35
up vote 3 down vote

Call setrlimit yourself or use softlimit or something like it.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.