6 The beanstalk protocol runs over TCP using ASCII encoding. Clients connect,
7 send commands and data, wait for responses, and close the connection. For each
8 connection, the server processes commands serially in the order in which they
9 were received and sends responses in the same order. All integers in the
10 protocol are formatted in decimal and (unless otherwise indicated)
13 Names, in this protocol, are ASCII strings. They may contain letters (A-Z and
14 a-z), numerals (0-9), hyphen ("-"), plus ("+"), slash ("/"), semicolon (";"),
15 dot ("."), dollar-sign ("$"), underscore ("_"), and parentheses ("(" and ")"),
16 but they may not begin with a hyphen. They are terminated by white space
17 (either a space char or end of line). Each name must be at least one character
20 The protocol contains two kinds of data: text lines and unstructured chunks of
21 data. Text lines are used for client commands and server responses. Chunks are
22 used to transfer job bodies and stats information. Each job body is an opaque
23 sequence of bytes. The server never inspects or modifies a job body and always
24 sends it back in its original form. It is up to the clients to agree on a
25 meaningful interpretation of job bodies.
27 The client may issue the "quit" command, or simply close the TCP connection
28 when it no longer has use for the server. However, beanstalkd performs very
29 well with a large number of open connections, so it is usually better for the
30 client to keep its connection open and reuse it as much as possible. This also
31 avoids the overhead of establishing new TCP connections.
33 If a client violates the protocol (such as by sending a request that is not
34 well-formed or a command that does not exist) or if the server has an error,
35 the server will reply with one of the following error messages:
37 - "OUT_OF_MEMORY\r\n" The server cannot allocate enough memory for the job.
38 The client should try again later.
40 - "INTERNAL_ERROR\r\n" This indicates a bug in the server. It should never
41 happen. If it does happen, please report it at
42 http://groups.google.com/group/beanstalk-talk.
44 - "BAD_FORMAT\r\n" The client sent a command line that was not well-formed.
45 This can happen if the line does not end with \r\n, if non-numeric
46 characters occur where an integer is expected, if the wrong number of
47 arguments are present, or if the command line is mal-formed in any other
50 - "UNKNOWN_COMMAND\r\n" The client sent a command that the server does not
53 These error responses will not be listed in this document for individual
54 commands in the following sections, but they are implicitly included in the
55 description of all commands. Clients should be prepared to receive an error
56 response after any command.
58 As a last resort, if the server has a serious error that prevents it from
59 continuing service to the current client, the server will close the
65 A job in beanstalk gets created by a client with the "put" command. During its
66 life it can be in one of four states: "ready", "reserved", "delayed", or
67 "buried". After the put command, a job typically starts out ready. It waits in
68 the ready queue until a worker comes along and runs the "reserve" command. If
69 this job is next in the queue, it will be reserved for the worker. The worker
70 will execute the job; when it is finished the worker will send a "delete"
71 command to delete the job.
73 Here is a picture of the typical job lifecycle:
77 -----> [READY] ---------> [RESERVED] --------> *poof*
81 Here is a picture with more possibilities:
85 put with delay release with delay
86 ----------------> [DELAYED] <------------.
90 put v reserve | delete
91 -----------------> [READY] ---------> [RESERVED] --------> *poof*
99 [BURIED] <---------------'
105 The system has one or more tubes. Each tube consists of a ready queue and a
106 delay queue. Each job spends its entire life in one tube. Consumers can show
107 interest in tubes by sending the "watch" command; they can show disinterest by
108 sending the "ignore" command. This set of interesting tubes is said to be a
109 consumer's "watch list". When a client reserves a job, it may come from any of
110 the tubes in its watch list.
112 When a client connects, its watch list is initially just the tube named
113 "default". If it submits jobs without having sent a "use" command, they will
114 live in the tube named "default".
116 Tubes are created on demand whenever they are referenced. If a tube is empty
117 (that is, it contains no ready, delayed, or buried jobs) and no client refers
118 to it, it will be deleted.
123 The "put" command is for any process that wants to insert a job into the queue.
124 It comprises a command line followed by the job body:
126 put <pri> <delay> <ttr> <bytes>\r\n
129 It inserts a job into the client's currently used tube (see the "use" command
132 - <pri> is an integer < 2**32. Jobs with smaller priority values will be
133 scheduled before jobs with larger priorities. The most urgent priority is 0;
134 the least urgent priority is 4,294,967,295.
136 - <delay> is an integer number of seconds to wait before putting the job in
137 the ready queue. The job will be in the "delayed" state during this time.
139 - <ttr> -- time to run -- is an integer number of seconds to allow a worker
140 to run this job. This time is counted from the moment a worker reserves
141 this job. If the worker does not delete, release, or bury the job within
142 <ttr> seconds, the job will time out and the server will release the job.
143 The minimum ttr is 1. If the client sends 0, the server will silently
144 increase the ttr to 1.
146 - <bytes> is an integer indicating the size of the job body, not including the
147 trailing "\r\n". This value must be less than max-job-size (default: 2**16).
149 - <data> is the job body -- a sequence of bytes of length <bytes> from the
152 After sending the command line and body, the client waits for a reply, which
155 - "INSERTED <id>\r\n" to indicate success.
157 - <id> is the integer id of the new job
159 - "BURIED <id>\r\n" if the server ran out of memory trying to grow the
160 priority queue data structure.
162 - <id> is the integer id of the new job
164 - "EXPECTED_CRLF\r\n" The job body must be followed by a CR-LF pair, that is,
165 "\r\n". These two bytes are not counted in the job size given by the client
166 in the put command line.
168 - "JOB_TOO_BIG\r\n" The client has requested to put a job with a body larger
169 than max-job-size bytes.
171 - "DRAINING\r\n" This means that the server has been put into "drain mode" and
172 is no longer accepting new jobs. The client should try another server or
173 disconnect and try again later. To put the server in drain mode, send the
174 SIGUSR1 signal to the process.
176 The "use" command is for producers. Subsequent put commands will put jobs into
177 the tube specified by this command. If no use command has been issued, jobs
178 will be put into the tube named "default".
182 - <tube> is a name at most 200 bytes. It specifies the tube to use. If the
183 tube does not exist, it will be created.
189 - <tube> is the name of the tube now being used.
194 A process that wants to consume jobs from the queue uses "reserve", "delete",
195 "release", and "bury". The first worker command, "reserve", looks like this:
199 Alternatively, you can specify a timeout as follows:
201 reserve-with-timeout <seconds>\r\n
203 This will return a newly-reserved job. If no job is available to be reserved,
204 beanstalkd will wait to send a response until one becomes available. Once a
205 job is reserved for the client, the client has limited time to run (TTR) the
206 job before the job times out. When the job times out, the server will put the
207 job back into the ready queue. Both the TTR and the actual time left can be
208 found in response to the stats-job command.
210 If more than one job is ready, beanstalkd will choose the one with the
211 smallest priority value. Within each priority, it will choose the one that
214 A timeout value of 0 will cause the server to immediately return either a
215 response or TIMED_OUT. A positive value of timeout will limit the amount of
216 time the client will block on the reserve request until a job becomes
219 During the TTR of a reserved job, the last second is kept by the server as a
220 safety margin, during which the client will not be made to wait for another
221 job. If the client issues a reserve command during the safety margin, or if
222 the safety margin arrives while the client is waiting on a reserve command,
223 the server will respond with:
227 This gives the client a chance to delete or release its reserved job before
228 the server automatically releases it.
232 If a non-negative timeout was specified and the timeout exceeded before a job
233 became available, or if the client's connection is half-closed, the server
234 will respond with TIMED_OUT.
236 Otherwise, the only other response to this command is a successful reservation
237 in the form of a text line followed by the job body:
239 RESERVED <id> <bytes>\r\n
242 - <id> is the job id -- an integer unique to this job in this instance of
245 - <bytes> is an integer indicating the size of the job body, not including
248 - <data> is the job body -- a sequence of bytes of length <bytes> from the
249 previous line. This is a verbatim copy of the bytes that were originally
250 sent to the server in the put command for this job.
252 The delete command removes a job from the server entirely. It is normally used
253 by the client when the job has successfully run to completion. A client can
254 delete jobs that it has reserved, ready jobs, delayed jobs, and jobs that are
255 buried. The delete command looks like this:
259 - <id> is the job id to delete.
261 The client then waits for one line of response, which may be:
263 - "DELETED\r\n" to indicate success.
265 - "NOT_FOUND\r\n" if the job does not exist or is not either reserved by the
266 client, ready, or buried. This could happen if the job timed out before the
267 client sent the delete command.
269 The release command puts a reserved job back into the ready queue (and marks
270 its state as "ready") to be run by any client. It is normally used when the job
271 fails because of a transitory error. It looks like this:
273 release <id> <pri> <delay>\r\n
275 - <id> is the job id to release.
277 - <pri> is a new priority to assign to the job.
279 - <delay> is an integer number of seconds to wait before putting the job in
280 the ready queue. The job will be in the "delayed" state during this time.
282 The client expects one line of response, which may be:
284 - "RELEASED\r\n" to indicate success.
286 - "BURIED\r\n" if the server ran out of memory trying to grow the priority
287 queue data structure.
289 - "NOT_FOUND\r\n" if the job does not exist or is not reserved by the client.
291 The bury command puts a job into the "buried" state. Buried jobs are put into a
292 FIFO linked list and will not be touched by the server again until a client
293 kicks them with the "kick" command.
295 The bury command looks like this:
299 - <id> is the job id to release.
301 - <pri> is a new priority to assign to the job.
303 There are two possible responses:
305 - "BURIED\r\n" to indicate success.
307 - "NOT_FOUND\r\n" if the job does not exist or is not reserved by the client.
309 The "touch" command allows a worker to request more time to work on a job.
310 This is useful for jobs that potentially take a long time, but you still want
311 the benefits of a TTR pulling a job away from an unresponsive worker. A worker
312 may periodically tell the server that it's still alive and processing a job
313 (e.g. it may do this on DEADLINE_SOON). The command postpones the auto
314 release of a reserved job until TTR seconds from when the command is issued.
316 The touch command looks like this:
320 - <id> is the ID of a job reserved by the current connection.
322 There are two possible responses:
324 - "TOUCHED\r\n" to indicate success.
326 - "NOT_FOUND\r\n" if the job does not exist or is not reserved by the client.
328 The "watch" command adds the named tube to the watch list for the current
329 connection. A reserve command will take a job from any of the tubes in the
330 watch list. For each new connection, the watch list initially consists of one
331 tube, named "default".
335 - <tube> is a name at most 200 bytes. It specifies a tube to add to the watch
336 list. If the tube doesn't exist, it will be created.
342 - <count> is the integer number of tubes currently in the watch list.
344 The "ignore" command is for consumers. It removes the named tube from the
345 watch list for the current connection.
351 - "WATCHING <count>\r\n" to indicate success.
353 - <count> is the integer number of tubes currently in the watch list.
355 - "NOT_IGNORED\r\n" if the client attempts to ignore the only tube in its
361 The peek commands let the client inspect a job in the system. There are four
362 variations. All but the first operate only on the currently used tube.
364 - "peek <id>\r\n" - return job <id>.
366 - "peek-ready\r\n" - return the next ready job.
368 - "peek-delayed\r\n" - return the delayed job with the shortest delay left.
370 - "peek-buried\r\n" - return the next job in the list of buried jobs.
372 There are two possible responses, either a single line:
374 - "NOT_FOUND\r\n" if the requested job doesn't exist or there are no jobs in
377 Or a line followed by a chunk of data, if the command was successful:
379 FOUND <id> <bytes>\r\n
382 - <id> is the job id.
384 - <bytes> is an integer indicating the size of the job body, not including
387 - <data> is the job body -- a sequence of bytes of length <bytes> from the
390 The kick command applies only to the currently used tube. It moves jobs into
391 the ready queue. If there are any buried jobs, it will only kick buried jobs.
392 Otherwise it will kick delayed jobs. It looks like:
396 - <bound> is an integer upper bound on the number of jobs to kick. The server
397 will kick no more than <bound> jobs.
399 The response is of the form:
403 - <count> is an integer indicating the number of jobs actually kicked.
405 The kick-job command is a variant of kick that operates with a single job
406 identified by its job id. If the given job id exists and is in a buried or
407 delayed state, it will be moved to the ready queue of the the same tube where it
408 currently belongs. The syntax is:
412 - <id> is the job id to kick.
414 The response is one of:
416 - "NOT_FOUND\r\n" if the job does not exist or is not in a kickable state. This
417 can also happen upon internal errors.
419 - "KICKED\r\n" when the operation succeeded.
421 The stats-job command gives statistical information about the specified job if
422 it exists. Its form is:
428 The response is one of:
430 - "NOT_FOUND\r\n" if the job does not exist.
432 - "OK <bytes>\r\n<data>\r\n"
434 - <bytes> is the size of the following data section in bytes.
436 - <data> is a sequence of bytes of length <bytes> from the previous line. It
437 is a YAML file with statistical information represented a dictionary.
439 The stats-job data is a YAML file representing a single dictionary of strings
440 to scalars. It contains these keys:
444 - "tube" is the name of the tube that contains this job
446 - "state" is "ready" or "delayed" or "reserved" or "buried"
448 - "pri" is the priority value set by the put, release, or bury commands.
450 - "age" is the time in seconds since the put command that created this job.
452 - "delay" is the integer number of seconds to wait before putting this job in
455 - "ttr" -- time to run -- is the integer number of seconds a worker is
456 allowed to run this job.
458 - "time-left" is the number of seconds left until the server puts this job
459 into the ready queue. This number is only meaningful if the job is
460 reserved or delayed. If the job is reserved and this amount of time
461 elapses before its state changes, it is considered to have timed out.
463 - "file" is the number of the earliest binlog file containing this job.
464 If -b wasn't used, this will be 0.
466 - "reserves" is the number of times this job has been reserved.
468 - "timeouts" is the number of times this job has timed out during a
471 - "releases" is the number of times a client has released this job from a
474 - "buries" is the number of times this job has been buried.
476 - "kicks" is the number of times this job has been kicked.
478 The stats-tube command gives statistical information about the specified tube
479 if it exists. Its form is:
481 stats-tube <tube>\r\n
483 - <tube> is a name at most 200 bytes. Stats will be returned for this tube.
485 The response is one of:
487 - "NOT_FOUND\r\n" if the tube does not exist.
489 - "OK <bytes>\r\n<data>\r\n"
491 - <bytes> is the size of the following data section in bytes.
493 - <data> is a sequence of bytes of length <bytes> from the previous line. It
494 is a YAML file with statistical information represented a dictionary.
496 The stats-tube data is a YAML file representing a single dictionary of strings
497 to scalars. It contains these keys:
499 - "name" is the tube's name.
501 - "current-jobs-urgent" is the number of ready jobs with priority < 1024 in
504 - "current-jobs-ready" is the number of jobs in the ready queue in this tube.
506 - "current-jobs-reserved" is the number of jobs reserved by all clients in
509 - "current-jobs-delayed" is the number of delayed jobs in this tube.
511 - "current-jobs-buried" is the number of buried jobs in this tube.
513 - "total-jobs" is the cumulative count of jobs created in this tube in
514 the current beanstalkd process.
516 - "current-using" is the number of open connections that are currently
519 - "current-waiting" is the number of open connections that have issued a
520 reserve command while watching this tube but not yet received a response.
522 - "current-watching" is the number of open connections that are currently
525 - "pause" is the number of seconds the tube has been paused for.
527 - "cmd-delete" is the cumulative number of delete commands for this tube
529 - "cmd-pause-tube" is the cumulative number of pause-tube commands for this
532 - "pause-time-left" is the number of seconds until the tube is un-paused.
534 The stats command gives statistical information about the system as a whole.
539 The server will respond:
544 - <bytes> is the size of the following data section in bytes.
546 - <data> is a sequence of bytes of length <bytes> from the previous line. It
547 is a YAML file with statistical information represented a dictionary.
549 The stats data for the system is a YAML file representing a single dictionary
550 of strings to scalars. Entries described as "cumulative" are reset when the
551 beanstalkd process starts; they are not stored on disk with the -b flag.
553 - "current-jobs-urgent" is the number of ready jobs with priority < 1024.
555 - "current-jobs-ready" is the number of jobs in the ready queue.
557 - "current-jobs-reserved" is the number of jobs reserved by all clients.
559 - "current-jobs-delayed" is the number of delayed jobs.
561 - "current-jobs-buried" is the number of buried jobs.
563 - "cmd-put" is the cumulative number of put commands.
565 - "cmd-peek" is the cumulative number of peek commands.
567 - "cmd-peek-ready" is the cumulative number of peek-ready commands.
569 - "cmd-peek-delayed" is the cumulative number of peek-delayed commands.
571 - "cmd-peek-buried" is the cumulative number of peek-buried commands.
573 - "cmd-reserve" is the cumulative number of reserve commands.
575 - "cmd-use" is the cumulative number of use commands.
577 - "cmd-watch" is the cumulative number of watch commands.
579 - "cmd-ignore" is the cumulative number of ignore commands.
581 - "cmd-delete" is the cumulative number of delete commands.
583 - "cmd-release" is the cumulative number of release commands.
585 - "cmd-bury" is the cumulative number of bury commands.
587 - "cmd-kick" is the cumulative number of kick commands.
589 - "cmd-stats" is the cumulative number of stats commands.
591 - "cmd-stats-job" is the cumulative number of stats-job commands.
593 - "cmd-stats-tube" is the cumulative number of stats-tube commands.
595 - "cmd-list-tubes" is the cumulative number of list-tubes commands.
597 - "cmd-list-tube-used" is the cumulative number of list-tube-used commands.
599 - "cmd-list-tubes-watched" is the cumulative number of list-tubes-watched
602 - "cmd-pause-tube" is the cumulative number of pause-tube commands.
604 - "job-timeouts" is the cumulative count of times a job has timed out.
606 - "total-jobs" is the cumulative count of jobs created.
608 - "max-job-size" is the maximum number of bytes in a job.
610 - "current-tubes" is the number of currently-existing tubes.
612 - "current-connections" is the number of currently open connections.
614 - "current-producers" is the number of open connections that have each
615 issued at least one put command.
617 - "current-workers" is the number of open connections that have each issued
618 at least one reserve command.
620 - "current-waiting" is the number of open connections that have issued a
621 reserve command but not yet received a response.
623 - "total-connections" is the cumulative count of connections.
625 - "pid" is the process id of the server.
627 - "version" is the version string of the server.
629 - "rusage-utime" is the cumulative user CPU time of this process in seconds
632 - "rusage-stime" is the cumulative system CPU time of this process in
633 seconds and microseconds.
635 - "uptime" is the number of seconds since this server process started running.
637 - "binlog-oldest-index" is the index of the oldest binlog file needed to
638 store the current jobs.
640 - "binlog-current-index" is the index of the current binlog file being
641 written to. If binlog is not active this value will be 0.
643 - "binlog-max-size" is the maximum size in bytes a binlog file is allowed
644 to get before a new binlog file is opened.
646 - "binlog-records-written" is the cumulative number of records written
649 - "binlog-records-migrated" is the cumulative number of records written
650 as part of compaction.
652 - "id" is a random id string for this server process, generated when each
653 beanstalkd process starts.
655 - "hostname" the hostname of the machine as determined by uname.
657 The list-tubes command returns a list of all existing tubes. Its form is:
666 - <bytes> is the size of the following data section in bytes.
668 - <data> is a sequence of bytes of length <bytes> from the previous line. It
669 is a YAML file containing all tube names as a list of strings.
671 The list-tube-used command returns the tube currently being used by the
680 - <tube> is the name of the tube being used.
682 The list-tubes-watched command returns a list tubes currently being watched by
683 the client. Its form is:
685 list-tubes-watched\r\n
692 - <bytes> is the size of the following data section in bytes.
694 - <data> is a sequence of bytes of length <bytes> from the previous line. It
695 is a YAML file containing watched tube names as a list of strings.
697 The quit command simply closes the connection. Its form is:
701 The pause-tube command can delay any new job being reserved for a given time. Its form is:
703 pause-tube <tube-name> <delay>\r\n
705 - <tube> is the tube to pause
707 - <delay> is an integer number of seconds < 2**32 to wait before reserving any more
710 There are two possible responses:
712 - "PAUSED\r\n" to indicate success.
714 - "NOT_FOUND\r\n" if the tube does not exist.