https://github.com/kiwanami/emacs-deferred.git
git clone 'git://github.com/kiwanami/emacs-deferred.git'
‘deferred.el’ provides facilities to manage asynchronous tasks. The API and implementations were translated from JSDeferred (by cho45) and Mochikit.Async (by Bob Ippolito) in JavaScript.
You can install deferred.el from MELPA by package.el.
You can find following sample codes in ‘deferred-sample.el’. Executing ‘eval-last-sexp’ (C-x C-e), you can try those codes.
This is a basic deferred chain. This code puts some outputs into message buffer, and then require a number from minibuffer.
Chain:
(deferred:$
(deferred:next
(lambda () (message "deferred start")))
(deferred:nextc it
(lambda ()
(message "chain 1")
1))
(deferred:nextc it
(lambda (x)
(message "chain 2 : %s" x)))
(deferred:nextc it
(lambda ()
(read-minibuffer "Input a number: ")))
(deferred:nextc it
(lambda (x)
(message "Got the number : %i" x)))
(deferred:error it
(lambda (err)
(message "Wrong input : %s" err))))After evaluating this s-exp and waiting for 1 second, a message is shown in the minibuffer.
Timer:
(deferred:$
(deferred:wait 1000) ; 1000msec
(deferred:nextc it
(lambda (x)
(message "Timer sample! : %s msec" x))))This s-exp inserts the result that is performed by the command ‘ls -la’. (This s-exp may not run in windows. Try ‘dir’ command.)
Command process:
(deferred:$
(deferred:process "ls" "-la")
(deferred:nextc it
(lambda (x) (insert x))))This s-exp inserts a text from http://www.gnu.org asynchronously. (You can clear the result with undo command.)
HTTP GET:
(require 'url)
(deferred:$
(deferred:url-retrieve "http://www.gnu.org")
(deferred:nextc it
(lambda (buf)
(insert (with-current-buffer buf (buffer-string)))
(kill-buffer buf))))This s-exp inserts an image from google asynchronously.
Get an image:
(deferred:$
(deferred:url-retrieve "http://www.google.co.jp/intl/en_com/images/srpr/logo1w.png")
(deferred:nextc it
(lambda (buf)
(insert-image
(create-image
(let ((data (with-current-buffer buf (buffer-string))))
(substring data (+ (string-match "\n\n" data) 2)))
'png t))
(kill-buffer buf))))This s-exp retrieves two images from google concurrently and wait for the both results. Then, the file sizes of the images are inserted the current buffer.
Parallel deferred:
(deferred:$
(deferred:parallel
(lambda ()
(deferred:url-retrieve "http://www.google.co.jp/intl/en_com/images/srpr/logo1w.png"))
(lambda ()
(deferred:url-retrieve "http://www.google.co.jp/images/srpr/nav_logo14.png")))
(deferred:nextc it
(lambda (buffers)
(loop for i in buffers
do
(insert
(format
"size: %s\n"
(with-current-buffer i (length (buffer-string)))))
(kill-buffer i)))))This s-exp executes following tasks: * Getting an image by wget command, * Resizing the image by convert command in ImageMagick, * Insert the re-sized image into the current buffer. You can construct the control structure of deferred tasks, like try-catch-finally in Java.
Get an image by wget and resize by ImageMagick:
(deferred:$
;; try
(deferred:$
(deferred:process "wget" "-O" "a.jpg" "http://www.gnu.org/software/emacs/tour/images/splash.png")
(deferred:nextc it
(lambda () (deferred:process "convert" "a.jpg" "-resize" "100x100" "jpg:b.jpg")))
(deferred:nextc it
(lambda ()
(clear-image-cache)
(insert-image (create-image (expand-file-name "b.jpg") 'jpeg nil)))))
;; catch
(deferred:error it ;
(lambda (err)
(insert "Can not get a image! : " err)))
;; finally
(deferred:nextc it
(lambda ()
(deferred:parallel
(lambda () (delete-file "a.jpg"))
(lambda () (delete-file "b.jpg")))))
(deferred:nextc it
(lambda (x) (message ">> %s" x))))Here is an another sample code for try-catch-finally blocks. This is simpler than above code because of the ‘deferred:try’ macro. (Note: They bring the same results practically, but are not perfectly identical. The ‘finally’ task may not be called because of asynchrony.)
Try-catch-finally:
(deferred:$
(deferred:try
(deferred:$
(deferred:process "wget" "-O" "a.jpg" "http://www.gnu.org/software/emacs/tour/images/splash.png")
(deferred:nextc it
(lambda () (deferred:process "convert" "a.jpg" "-resize" "100x100" "jpg:b.jpg")))
(deferred:nextc it
(lambda ()
(clear-image-cache)
(insert-image (create-image (expand-file-name "b.jpg") 'jpeg nil)))))
:catch
(lambda (err) (insert "Can not get a image! : " err))
:finally
(lambda ()
(delete-file "a.jpg")
(delete-file "b.jpg")))
(deferred:nextc it
(lambda (x) (message ">> %s" x))))Although a long time command is executed (3 second sleeping), the task is rejected by timeout for 1 second.
The function deferred:earlier also runs asynchronous tasks concurrently, however, the next deferred task receives the first result. The other results and tasks will be rejected (canceled or ignored).
Timeout Process:
(deferred:$
(deferred:earlier
(deferred:process "sh" "-c" "sleep 3 | echo 'hello!'")
(deferred:$
(deferred:wait 1000) ; timeout msec
(deferred:nextc it (lambda () "canceled!"))))
(deferred:nextc it
(lambda (x) (insert x))))Here is an another sample code for timeout, employing ‘deferred:timeout’ macro.
Timeout macro:
(deferred:$
(deferred:timeout
1000 "canceled!"
(deferred:process "sh" "-c" "sleep 3 | echo 'hello!'"))
(deferred:nextc it
(lambda (x) (insert x))))Note that the deferred:timeout and deferred:earlier just rejects the task result and does not stop the running task chains. Please see the document for deferred:cancel.
This s-exp plays an animation at the cursor position for few seconds. Then, you can move cursor freely, because the animation does not block Emacs.
Returning a deferred object in the deferred tasks, the returned task is executed before the next deferred one that is statically connected on the source code. (In this case, the interrupt task is dynamically connected.)
Employing a recursive structure of deferred tasks, you can construct a deferred loop. It may seem the multi-thread in Emacs Lisp.
Loop and animation:
(lexical-let ((count 0) (anm "-/|\\-")
(end 50) (pos (point))
(wait-time 50))
(deferred:$
(deferred:next
(lambda (x) (message "Animation started.")))
(deferred:nextc it
(deferred:lambda (x)
(save-excursion
(when (< 0 count)
(goto-char pos) (delete-char 1))
(insert (char-to-string
(aref anm (% count (length anm))))))
(if (> end (incf count)) ; return nil to stop this loop
(deferred:nextc (deferred:wait wait-time) self)))) ; return the deferred
(deferred:nextc it
(lambda (x)
(save-excursion
(goto-char pos) (delete-char 1))
(message "Animation finished.")))))Let's say you have an asynchronous function which takes a callback. For example, dbus.el, xml-rpc.el and websocket.el has such kind of asynchronous APIs. To use such libraries with deferred.el, you can make an unregistered deferred object using deferred:new and then start the deferred callback queue using deferred:callback-post in the callback given to the asynchronous function. If the asynchronous function supports “errorback”, you can use deferred:errorback-post to pass the error information to the following callback queue.
In the following example, run-at-time is used as an example for the asynchronous function. Deferred.el already has deferred:wait for this purpose so that you don't need the following code if you want to use run-at-time.
(deferred:$
(deferred:next
(lambda ()
(message "1")
1))
(deferred:nextc it
(lambda (x)
(lexical-let ((d (deferred:new #'identity)))
(run-at-time 0 nil (lambda (x)
;; Start the following callback queue now.
(deferred:callback-post d x))
x)
;; Return the unregistered (not yet started) callback
;; queue, so that the following queue will wait until it
;; is started.
d)))
;; You can connect deferred callback queues
(deferred:nextc it
(lambda (x)
(message "%s" (1+ x)))))deferred:next (callback)
deferred:nextc (d callback)
deferred:error (d errorback)
If this function does not throw an error, the subsequent callback functions are executed.
deferred:cancel (d)
Invalidate the given deferred object.
Because this function modifies the deferred object, one can not used the given deferred instance again.
This function just cancels the given deferred instance, not the whole deferred chain. In the current deferred implementation, a message of cancellation can not propagate to chained deferred objects because the chain is built by the singly linked list. If the deferred chains may be canceled on your code, you should care the side-effect tasks.
deferred:watch (d callback)
Create a deferred object with watch task and connect it to the given deferred object.
The watch task CALLBACK can not affect deferred chains with return values.
This function is used in following purposes, simulation of try-finally block in asynchronous tasks, monitoring of progress of deferred tasks.
deferred:wait (msec)
Return a deferred object that will be called after the specified millisecond.
The subsequent deferred task receives the actual elapse time in millisecond.
deferred:$
Arguments / more than one deferred forms
Return / the last deferred object
deferred:loop (number-or-list callback)
If a list is given, not a number, the function visits each elements in the list like ‘mapc’.
deferred:parallel (list-or-alist)
Return a deferred object that executes given functions in parallel and wait for all callback values.
The subsequent deferred task receives a list of the results. The order of the results is corresponding to one of the argument.
Giving an alist of tasks as the argument, the results alist is returned.
If the parallel task throws an error, the error object is passed as a result.
deferred:earlier (list-or-alist)
deferred:cancel)Giving an alist of tasks as the argument, a cons cell is returned as a result.
deferred:call (function args…)
a wrapper of the function ‘funcall’
deferred:apply (function args)
a wrapper of the function ‘apply’
deferred:process (command args…) / deferred:process-shell (command args…)
Execute a command asynchronously. These functions are wrappers of ‘start-process’ and ‘start-process-shell-command’.
The subsequent deferred task receives the stdout from the command as a string.
deferred:process-buffer (command args…) / deferred:process-shell-buffer (command args…)
Execute a command asynchronously. These functions are wrappers of ‘start-process’ and ‘start-process-shell-command’.
deferred:wait-idle (msec)
Return a deferred object that will be called when Emacs has been idle for the specified millisecond.
The subsequent deferred task receives the elapse time in millisecond.
deferred:url-retrieve (url [cbargs])
A wrapper function of ‘url-retrieve’ in the ‘url’ package.
[experimental] deferred:url-get (url [params])
[experimental] deferred:url-post (url [params])
deferred:new ([callback])
Create a deferred object
The created deferred object is never called until someone call the function ‘deferred:callback’ or ‘deferred:errorback’.
Using this object, a deferred chain can pause to wait for other events. (See the source for ‘deferred:wait’.)
deferred:succeed ([value])
Create a deferred object that has been called the callback function.
When a deferred task is connected, the subsequent task will be executed immediately (synchronously).
deferred:fail ([error])
Create a deferred object that has been called the errorback function.
When a deferred task is connected, the subsequent task will be executed immediately (synchronously).
deferred:callback (d [value])
Start executing the deferred tasks. The first task is executed synchronously.
deferred:callback-post (d [value])
Start executing the deferred tasks. The first task is executed asynchronously.
deferred:errorback (d [error])
Start executing the deferred tasks from errorback. The first task is executed synchronously.
deferred:errorback-post (d [error])
Start executing the deferred tasks from errorback. The first task is executed asynchronously.
deferred:try (d &key catch finally)
Try-catch-finally macro. This macro simulates the try-catch-finally block asynchronously.
Because of asynchrony, this macro does not ensure that the ‘finally’ task should be called.
This macro is implemented by ‘deferred:error’ and ‘deferred:watch’.
deferred:timeout (msec timeout-form d)
Time out macro on a deferred task ‘d’.
If the deferred task ‘d’ does not complete within ‘timeout-msec’, this macro rejects the deferred task and return the ‘timeout-form’. (See the document for deferred:cancel)
This macro is implemented by ‘deferred:earlier’ and ‘deferred:wait’.
deferred:process…
deferred:processc (d command args…)
deferred:process-bufferc (d command args…)
deferred:process-shellc (d command args…)
deferred:process-shell-bufferc (d command args…)
This macro wraps the deferred:process function in deferred:nextc and connect the given deferred task.
Some deferred functions can fire a deferred chain implicitly. Following functions register a deferred object with the execution queue to run asynchronously.
The deferred tasks those are created by ‘deferred:new’ are never called. Using this object, a deferred chain can pause to wait for other events. (See the source for ‘deferred:wait’.)
One can fire the chain before connecting. That is, deferred objects wait for connecting the subsequent task holding the result value. The functions ‘deferred:succeed’ and ‘deferred:fail’ create those waiting objects.
The ‘static connection (statically connected)’ is a connection between deferred tasks on the source code. This is a basic usage for the deferred chain.
The static connection is almost equivalent to ordinary callback notation as an argument in the function declarations. The deferred notation is easy to read and write better than the callback one, because the sequence of asynchronous tasks can be written by the deferred notation straightforward.
Returning a deferred object in the deferred tasks, the returned task is executed before the next deferred one that is statically connected on the source code. This is the ‘dynamic connection (dynamically connected)’.
Employing a recursive structure of deferred tasks, you can construct higher level control structures, such as loop.
Some discussions of writing deferred codes.
Using the lexical scope macro, such as ‘lexical-let’, the deferred tasks defined by lambdas can access local variables.
lexical-let Ex.:
(lexical-let ((a (point)))
(deferred:$
(deferred:wait 1000)
(deferred:nextc it
(lambda (x)
(goto-char a)
(insert "here!")))))If you write a code of deferred tasks without lexical scope macros, you should be careful with the scopes of each variables.
The ‘excursion’ functions those hold the current status with the s-exp form, such as ‘save-execursion’ or ‘with-current-buffer’, are not valid in the deferred tasks, because of execution asynchronously.
Wrong Ex.:
(with-current-buffer (get-buffer "*Message*")
(deferred:$
(deferred:wait 1000)
(deferred:nextc it
(lambda (x)
(insert "Time: %s " x) ; 'insert' may not be in the *Message* buffer!
))))In this case, using lexical scope macros to access the buffer variable, you can change the buffer in the deferred task.
Corrected:
(lexical-let ((buf (get-buffer "*Message*")))
(deferred:$
(deferred:wait 1000)
(deferred:nextc it
(lambda (x)
(with-current-buffer buf ; Set buffer in the asynchronous task.
(insert "Time: %s " x))))))However the dynamic connection is a powerful feature, sometimes it causes bugs of the wrong execution order, because of returning not intended deferred objects.
Then, you should watch the return values of the deferred tasks not to cause an unexpected dynamic connection.
The debugging of asynchronous tasks is difficult. Of course, you can use debugger for deferred tasks, but asynchronous tasks cause some troubles, such as interruptions of your debugging and timing gap of simultaneous deferred tasks. Therefore, logging is a safe debugging to observe the tasks correctly, for example, using the ‘message’ function and making custom application log buffer.
If deferred tasks fall into an infinite loop unexpectedly (but Emacs may not freeze), calling the command ‘deferred:clear-queue’, you can stop the deferred tasks immediately.
If the errors occurred in deferred tasks are caught by no errorback functions, finally the deferred framework catches it and reports to the message buffer. Because the implementation of the framework uses a ‘condition-case’ form, the debugger can not catch the signals normally. If you want to debug the errors in the deferred tasks with the debug-on-error mechanism, set the variable ‘deferred:debug-on-signal’ non-nil.
Wrapping a deferred task in the function ‘deferred:sync!’, you can wait for the result of the task synchronously. However, the wrapper function should be used for test or debug purpose, because the synchronous waiting is not exact.
Writing deferred tasks with ‘deferred.el’, you may write a lot of ‘deferred:nextc’ and ‘lambda’ to define tasks. Defining a macro, you may write codes shortly. The test code ‘test-deferred.el’ uses many macros to shorten test codes.
On the other hand, using macros to hide ‘lambda’, it is difficult to realize when the deferred codes are evaluated. That is why ‘deferred.el’ does not provide lot of convenient macros. If you use macros, be careful evaluation timing of deferred forms.
Following documents are good introduction to deferred.
(C) 2010, 2011 SAKURAI Masashi All rights reserved. m.sakurai at kiwanami.net