1 <!DOCTYPE HTML PUBLIC
"-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
2 <html xmlns=
"http://www.w3.org/1999/xhtml">
5 <title>Postmodern
</title>
6 <link rel=
"stylesheet" type=
"text/css" href=
"style.css"/>
7 <meta http-equiv=
"Content-Type" content=
"text/html; charset=utf-8"/>
14 <p>Postmodern is a Common Lisp library for interacting with
<a
15 href=
"http://www.postgresql.org">PostgreSQL
</a> databases.
19 <li>Efficient communication with the database server without
20 need for foreign libraries.
</li>
21 <li>Support for UTF-
8 on Unicode-aware Lisp implementations
</li>
22 <li>A syntax for mixing SQL and Lisp code
</li>
23 <li>Convenient support for prepared statements and stored
25 <li>A metaclass for simple database-access objects
</li>
28 <p>The biggest differences between this library and
<a
29 href=
"http://clsql.b9.com/">CLSQL
</a>/CommonSQL are that
30 Postmodern has no intention of being portable across different SQL
31 implementations (it embraces non-standard PostgreSQL features), and
32 approaches extensions like lispy SQL and database access objects
33 in a quite different way. This library was written because the
34 CLSQL approach did not really work for me, your mileage may
40 <li><a href=
"#news">News
</a></li>
41 <li><a href=
"#dependencies">Dependencies
</a></li>
42 <li><a href=
"#license">License
</a></li>
43 <li><a href=
"#download">Download and installation
</a></li>
44 <li><a href=
"#support">Support and mailing lists
</a></li>
45 <li><a href=
"#quickstart">Quickstart
</a></li>
46 <li><a href=
"#reference">Reference
</a></li>
47 <li><a href=
"#caveats">Caveats and to-dos
</a></li>
48 <li><a href=
"#resources">Resources
</a></li>
51 <h2><a name=
"news"></a>News
</h2>
53 <p class=
"news"><em>28-
11-
2012:
</em> <a href=
"http://marijnhaverbeke.nl/postmodern/postmodern-1.19.tgz">Version
54 1.19</a>: Fix ECL incompatibilities,
55 add
<a href=
"postmodern.html#upsert-dao"><code>upsert-dao
</code></a>
57 for
<a href=
"cl-postgres.html#wait-for-notification">notifications
</a>
58 and
<a href=
"cl-postgres.html#bulk-copying">bulk copying
</a>, and
59 make unix sockets work on CCL.
</p>
61 <p class=
"news"><em>19-
10-
2011:
</em> <a href=
"http://marijnhaverbeke.nl/postmodern/postmodern-1.18.tgz">Version
62 1.18</a>: Improve support for array literals, add array slicing.
63 Switch test suite over
64 to
<a href=
"https://github.com/adlai/Eos">Eos
</a>.
65 Added
<a href=
"postmodern.html#make-dao"><code>make-dao
</code></a>,
<a href=
"postmodern.html#query"><code>:dao
</code></a>
67 and
<a href=
"postmodern.html#define-dao-finalization"><code>define-dao-finalization
</code></a>.
69 operators,
<a href=
"s-sql.html#for-update">locking
</a>,
70 and
<a href=
"s-sql.html#set-constraints">constraint deferring
</a>
72 a
<a href=
"postmodern.html#!unique"><code>!unique
</code></a>
73 operator for defining table constraints. Add
74 a
<a href=
"cl-postgres.html#database-error-constraint-name"><code>database-error-constraint-name
</code></a>
75 condition object accessor.
</p>
77 <p class=
"news"><em>02-
02-
2011:
</em> <a href=
"http://marijnhaverbeke.nl/postmodern/postmodern-1.17.tgz">Version
78 1.17</a>: Another modest maintenance release.
79 Fixes
<a href=
"simple-date.html#day-of-week"><code>day-of-week
</code></a>
81 the
<code>:plist
</code> <a href=
"postmodern.html#query">query
82 format
</a> actually work.
83 Adds
<a href=
"s-sql.html#sql-escape"><code>sql-escape
</code></a>
84 as an exported symbol for client code that needs to escape stuff.
85 Adds support for multi-dimensional arrays. Adds
86 the
<a href=
"postmodern.html#*ignore-unknown-columns*"><code>*ignore-unknown-columns*
</code></a>
87 configuration variable.
</p>
89 <p class=
"news"><em>02-
04-
2010:
</em> <a
90 href=
"http://marijnhaverbeke.nl/postmodern/postmodern-1.16.tgz">Version
91 1.16</a>: Introduces a
<code><a
92 href=
"postmodern.html#save-dao/transaction">save-dao/transaction
</a></code>,
93 because the old semantics were broken (originally inside of
94 transactions, after fixing that outside of them). Add support for
95 passing vectors as argument to prepared queries, and reading them
96 from query results. Add
<code>:on-update
</code> and
97 <code>:on-delete
</code> arguments to
<code><a
98 href=
"postmodern.html#!foreign">!foreign
</a></code>. Add
99 <code>:plist
</code> and
<code>:plists
</code> query result formats.
100 Guarantee that
<code><a
101 href=
"postmodern.html#deftable">deftable
</a></code> definitions
102 are executed in the order they were defined in. Moves the
<a
103 href=
"http://common-lisp.net/project/ieee-floats">ieee-floats
</a>
105 href=
"http://common-lisp.net/project/trivial-utf-8">trivial-utf-
8</a>
106 dependencies into the repository, so they don't have to separately
109 <p class=
"news"><em>02-
04-
2010:
</em> We are moving from common-lisp.net to
110 marijnhaverbeke.nl, and from darcs to git. New project home:
<a
111 href=
"http://marijnhaverbeke.nl/postmodern">http://marijnhaverbeke.nl/postmodern
</a>.
</p>
113 <p class=
"news"><em>01-
03-
2009:
</em> <a
114 href=
"http://marijnhaverbeke.nl/postmodern/postmodern-1.15.tgz">Version
115 1.15</a>: Almost a year since the last release. Postmodern is
116 still alive—it is just
<em>so
</em> perfect it hardly needs updates
117 anymore. In this release: Stop depending on the usocket library in
118 Allegro CL and SBCL, just use the built-in socket bindings
119 instead. Allow connecting over a Unix socket in SBCL. Support
120 natural joins in
<code>:select
</code>. Add
<code>:if-exists
</code>
121 argument to
<code>:drop-...
</code> operators, add support for
122 <code>:add-column
</code> form to
<code>:alter-table
</code>. Add
123 <code>:between-symmetric
</code> operator. Introduce
"ghost slot"
124 hack to DAO system to support fetching OIDs. Extend
125 <code>doquery
</code> to also handle parameterised queries.
</p>
127 <p class=
"news"><em>07-
03-
2009:
</em> <a
128 href=
"http://marijnhaverbeke.nl/postmodern/postmodern-1.14.tgz">Version
129 1.14</a>: Some more syntax supported in
<a
130 href=
"s-sql.html">S-SQL
</a>, add
<a
131 href=
"cl-postgres.html#*silently-truncate-rationals*"><code>*silently-truncate-rationals*
</code></a>
133 href=
"cl-postgres.html#*query-callback*"><code>*query-callback*
</code></a>,
135 href=
"postmodern.html#dao-keys"><code>dao-keys
</code></a>, and fix
138 <p class=
"news"><em>28-
08-
2008:
</em> <a
139 href=
"http://marijnhaverbeke.nl/postmodern/postmodern-1.13.tgz">Version
140 1.13</a>: A few small fixes and optimisations, and there is once
142 href=
"postmodern.html#deftable"><code>deftable
</code></a> ―
143 though it has a different role than it used to have.
</p>
145 <p class=
"news"><em>30-
04-
2008:
</em> <a
146 href=
"http://marijnhaverbeke.nl/postmodern/postmodern-1.12.tgz">Version
147 1.12</a>: Restores the
<a
148 href=
"postmodern.html#save-dao"><code>save-dao
</code></a> function
149 (though you should still be careful with it), adds
<a
150 href=
"cl-postgres.html#open-database">support
</a> <a
151 href=
"postmodern.html#*default-use-ssl*">for
</a> <a
152 href=
"http://www.postgresql.org/docs/current/static/ssl-tcp.html">SSL
</a>
153 connections, makes some error messages clearer, adds some S-SQL
<a
154 href=
"s-sql.html#between">operators
</a>, and fixes a few bugs.
</p>
156 <p class=
"news"><em>09-
04-
2008:
</em> <a
157 href=
"http://marijnhaverbeke.nl/postmodern/postmodern-1.11.tgz">Version
158 1.11</a>: Fixes bugs, most importantly one that caused
<a
159 href=
"cl-postgres.html">CL-postgres
</a> to blow up whenever a
<a
160 href=
"http://www.postgresql.org/docs/current/static/config-setting.html">connection
161 parameter
</a> was changed for an open connection. Also adds
<a
162 href=
"s-sql.html#*escape-sql-names-p*">automatic escaping
</a> of
163 reserved words to S-SQL, a
<a
164 href=
"postmodern.html#*max-pool-size*"><code>*max-pool-size*
</code></a>
166 href=
"postmodern.html#coalesce"><code>coalesce
</code></a> function
169 <p class=
"news"><em>19-
03-
2008:
</em> <a
170 href=
"http://marijnhaverbeke.nl/postmodern/postmodern-1.10.tgz">Version
171 1.10</a>: Note that this release is
<strong>not entirely
172 backwards-compatible
</strong>. It introduces a new, more flexible
173 approach to
<a href=
"postmodern.html#daos">DAO objects
</a>, a more
174 pluggable
<a href=
"cl-postgres.html#reading">handling
</a> of
175 custom data types, and a bunch of small clean-ups, optimizations,
176 and enhancements. See the
<a
177 href=
"migrating-to-1.10.html">migration guide
</a> for details.
</p>
179 <h2><a name=
"dependencies"></a>Dependencies
</h2>
181 <p>The library depends on
<a
182 href=
"http://cliki.net/usocket">usocket
</a> (except on SBCL and
183 ACL, where the built-in socket library is used),
<a
184 href=
"http://www.cliki.net/CL-MD5">md5
</a>,
<a
185 href=
"http://common-lisp.net/project/closer/closer-mop.html">closer-mop
</a>,
187 href=
"http://common-lisp.net/project/bordeaux-threads">bordeaux-threads
</a>
188 if you want thread-safe connection pools, and
<a
189 href=
"http://common-lisp.net/project/cl-plus-ssl/">CL+SSL
</a> when
190 SSL connections are needed.
</p>
192 <p>Postmodern itself is split into four different packages, some
193 of which can be used independently.
<a
194 href=
"simple-date.html">Simple-date
</a> is a very basic
195 implementation of date and time objects, used to support storing
196 and retrieving time-related SQL types.
<a
197 href=
"cl-postgres.html">CL-postgres
</a> is the low-level library
198 used for interfacing with a PostgreSQL server over a socket.
<a
199 href=
"s-sql.html">S-SQL
</a> is used to compile s-expressions to
200 strings of SQL code, escaping any Lisp values inside, and doing as
201 much as possible of the work at compile time. Finally,
<a
202 href=
"postmodern.html">Postmodern
</a> itself is the library that
203 tries to put all these things together into a convenient
204 programming interface.
</p>
206 <h2><a name=
"license"></a>License
</h2>
208 <p>Postmodern is released under a zlib-style license. Which
209 approximately means you can use the code in whatever way you like,
210 except for passing it off as your own or releasing a modified
211 version without indication that it is not the original.
</p>
213 <h2><a name=
"download"></a>Download and installation
</h2>
215 <p>The latest release of Postmodern can be downloaded from
<a
216 href=
"http://marijnhaverbeke.nl/postmodern/postmodern.tgz">http://marijnhaverbeke.nl/postmodern/postmodern.tgz
</a>,
218 href=
"http://www.cliki.net/ASDF-Install">asdf-install
</a>.
</p>
220 <p>A
<a href=
"http://git-scm.com/">git
</a> repository with the
221 most recent changes can be checked out with:
</p>
223 <pre>> git clone http://marijnhaverbeke.nl/git/postmodern
</pre>
225 <p>You can also view the repository on
<a
226 href=
"http://github.com/marijnh/Postmodern">github
</a>.
</p>
229 href=
"http://marijnhaverbeke.nl/postmodern/postmodern-latest.tgz">http://marijnhaverbeke.nl/postmodern/postmodern-latest.tgz
</a>
230 always contains a snapshot of the current repository head.
</p>
232 <h2><a name=
"support"></a>Support and mailing lists
</h2>
234 <p>The postmodern-devel can be used for questions, discussion,
235 bug-reports, patches, or anything else relating to this library.
236 To subscribe, send a message
237 to
<a href=
"mailto:postmodern-devel+subscribe@common-lisp.net">postmodern-devel+subscribe@common-lisp.net
</a>.
238 Or mail the author/maintainer
239 directly:
<a href=
"mailto:marijnh@gmail.com">Marijn
242 <h2><a name=
"quickstart"></a>Quickstart
</h2>
244 <p>This quickstart is intended to give you a feel of the way
245 coding with Postmodern works. Further details about the workings
246 of the library can be found in the
<a href=
"#reference">reference
249 <p>Assuming you have already installed it, first load and use the
253 (asdf:oos 'asdf:load-op :postmodern)
254 (use-package :postmodern)
</pre>
256 <p>If you have a PostgreSQL server running on localhost, with a
257 database called 'testdb' on it, which is accessible for user
258 'foucault' with password 'surveiller', you can connect like
262 (connect-toplevel
"testdb" "foucault" "surveiller" "localhost")
</pre>
264 <p>Which will establish a connection to be used by all code,
265 except for that wrapped in a
<a
266 href=
"postmodern.html#with-connection"><code>with-connection
</code></a>
267 form, which takes the same arguments but only establishes the
268 connection locally.
</p>
270 <p>Now for a basic sanity test:
</p>
273 (query
"select 22, 'Folie et déraison', 4.5")
274 ;; =
> ((
22 "Folie et déraison" 9/
2))
</pre>
276 <p>That should work.
<a href=
"postmodern.html#query">query
</a> is
277 the basic way to send queries to the database. The same query can
278 be expressed like this:
</p>
281 (query (:select
22 "Folie et déraison" 4.5))
282 ;; =
> ((
22 "Folie et déraison" 9/
2))
</pre>
284 <p>In many contexts, query strings and lists starting with
285 keywords can be used interchangeably. The lists will be compiled
286 to SQL. The
<a href=
"s-sql.html">S-SQL
</a> manual describes the
287 syntax used by these expressions. Lisp values occurring in them are
288 automatically escaped. In the above query, only constant values
289 are used, but it is possible to transparently use run-time values
293 (defun database-powered-addition (a b)
294 (query (:select (:+ a b)) :single))
295 (database-powered-addition
1030 204)
298 <p>That last argument,
<code>:single
</code>, indicates that we
299 want the result not as a list of lists (for the result rows), but
300 as a single value, since we know that we are only selecting one
301 value. Some other options are
<code>:rows
</code>,
302 <code>:row
</code>,
<code>:column
</code>,
<code>:alists
</code>, and
303 <code>:none
</code>. Their precise effect is documented in the
<a
304 href=
"postmodern.html#query">reference manual
</a>.
</p>
306 <p>You do not have to pull in the whole result of a query at once,
307 you can also iterate over it with the
<a
308 href=
"postmodern.html#doquery"><code>doquery
</code></a> macro:
</p>
311 (doquery (:select 'x 'y :from 'some-imaginary-table) (x y)
312 (format t
"On this row, x = ~A and y = ~A.~%" x y))
</pre>
314 <p>This is what a database-access class looks like:
</p>
318 ((name :col-type string :initarg :name
319 :reader country-name)
320 (inhabitants :col-type integer :initarg :inhabitants
321 :accessor country-inhabitants)
322 (sovereign :col-type (or db-null string) :initarg :sovereign
323 :accessor country-sovereign))
324 (:metaclass dao-class)
327 <p>The above defines a class that can be used to handle records in
328 a table with three columns: name, inhabitants, and sovereign. In
329 simple cases, the information above is enough to define the table
333 (dao-table-definition 'country)
334 ;; =
> "CREATE TABLE country (
335 ;; name TEXT NOT NULL,
336 ;; inhabitants INTEGER NOT NULL,
338 ;; PRIMARY KEY (name))"
339 (execute (dao-table-definition 'country))
</pre>
341 <p>This defines our table in the database.
<a
342 href=
"postmodern.html#execute">execute
</a> works like query, but
343 does not expect any results back.
</p>
345 <p>Let us add a few countries:
</p>
348 (insert-dao (make-instance 'country :name
"The Netherlands"
349 :inhabitants
16800000
350 :sovereign
"Willem-Alexander"))
351 (insert-dao (make-instance 'country :name
"Croatia"
352 :inhabitants
4400000))
</pre>
354 <p>Then, to update Croatia's population, we could do this:
</p>
357 (let ((croatia (get-dao 'country
"Croatia")))
358 (setf (country-inhabitants croatia)
4500000)
359 (update-dao croatia))
360 (query (:select '* :from 'country))
361 ;; =
> ((
"The Netherlands" 16800000 "Willem-Alexander")
362 ;; (
"Croatia" 4500000 :NULL))
</pre>
364 <p>Next, to demonstrate a bit more of the S-SQL syntax, here is
365 the query the utility function
<a
366 href=
"postmodern.html#list-tables"><code>list-tables
</code></a>
367 uses to get a list of the tables in a database:
</p>
370 (sql (:select 'relname :from 'pg-catalog.pg-class
371 :inner-join 'pg-catalog.pg-namespace :on (:= 'relnamespace 'pg-namespace.oid)
372 :where (:and (:= 'relkind
"r")
373 (:not-in 'nspname (:set
"pg_catalog" "pg_toast"))
374 (:pg-catalog.pg-table-is-visible 'pg-class.oid))))
375 ;; =
> "(SELECT relname FROM pg_catalog.pg_class
376 ;; INNER JOIN pg_catalog.pg_namespace ON (relnamespace = pg_namespace.oid)
377 ;; WHERE ((relkind = 'r') and (nspname NOT IN ('pg_catalog', 'pg_toast'))
378 ;; and pg_catalog.pg_table_is_visible(pg_class.oid)))"</pre>
380 <p><a href=
"s-sql.html#sql"><code>sql
</code></a> is a macro that
381 will simply compile a query, it can be useful for seeing how your
382 queries are expanded or if you want to do something unexpected
385 <p>As you can see, lists starting with keywords are used to
386 express SQL commands and operators (lists starting with something
387 else will be evaluated and then inserted into the query). Quoted
388 symbols name columns or tables (keywords can also be used but
389 might introduce ambiguities). The syntax supports subqueries,
390 multiple joins, stored procedures, etc. See the S-SQL
<a
391 href=
"s-sql.html">reference manual
</a> for a complete
394 <p>Finally, here is an example of the use of prepared
398 (defprepared sovereign-of
399 (:select 'sovereign :from 'country :where (:= 'name '$
1))
401 (sovereign-of
"The Netherlands")
402 ;; =
> "Willem-Alexander"</pre>
405 href=
"postmodern.html#defprepared"><code>defprepared
</code></a>
406 macro creates a function that takes the same amount of arguments
407 as there are
<code>$X
</code> placeholders in the given query. The
408 query will only be parsed and planned once (per database
409 connection), which can be faster, especially for complex
413 (disconnect-toplevel)
</pre>
415 <h2><a name=
"reference"></a>Reference
</h2>
417 <p>The reference manuals for the different components of
418 Postmodern are kept in separate files. For using the library in
419 the most straightforward way, you only really need to read the
<a
420 href=
"postmodern.html">Postmodern reference
</a> and glance over
421 the
<a href=
"s-sql.html">S-SQL reference
</a>. The
<a
422 href=
"simple-date.html">simple-date
</a> reference explains the
423 time-related data types included in Postmodern, and the
<a
424 href=
"cl-postgres.html">CL-postgres reference
</a> might be useful
425 if you just want a low-level library for talking to a PostgreSQL
429 <li><a href=
"postmodern.html">Postmodern
</a></li>
430 <li><a href=
"s-sql.html">S-SQL
</a></li>
431 <li><a href=
"simple-date.html">Simple-date
</a></li>
432 <li><a href=
"cl-postgres.html">CL-postgres
</a></li>
435 <h2><a name=
"caveats"></a>Caveats and to-dos
</h2>
439 <p><a href=
"simple-date.html">Simple-date
</a> has no concept of
440 time zones. This means that using it is rather error-prone, and if
441 you really need your time-keeping to be reliable and/or universal
442 you should either not use the types it provides or think really
443 hard about the way you handle time zones.
</p>
445 <p>Recently, a lot of work has been done on
<a
446 href=
"http://www.common-lisp.net/project/local-time">local-time
</a>,
447 which solves the same problem as simple-date, but does understand
449 href=
"http://common-lisp.net/cgi-bin/darcsweb/darcsweb.cgi?r=local-time-local-time-1.0">1.0
450 repository
</a> currently has code for integration with
451 CL-postgres, though this might not be stable yet.
</p>
455 <p>The Lisp code in Postmodern is theoretically portable across
456 implementations, and seems to work on all major ones. I am not
457 actively testing against new releases or obscure implementations,
458 but if you run into problems you are welcome to contact me through
460 href=
"mailto:postmodern-devel+subscribe@common-lisp.net">mailing
461 list
</a>, and we can try to solve them. Implementations that do
462 not have meta-object protocol support will not have DAOs, but all
463 other parts of the library should work (all widely used
464 implementations do support this).
</p>
466 <p>The library will definitely not work for PostgreSQL versions
467 older than
7.4 (it uses a client/server protocol that was
468 introduced in that version). On versions prior to
8.1, retrieving
469 date and time objects is broken, because their binary
470 representation was changed. Part of the functionality of
<a
471 href=
"postmodern.html#insert-dao"><code>insert-dao
</code></a>
472 (automatic defaulting of unbound slots) only works in PostgreSQL
475 <h3>Things that should be implemented
</h3>
477 <p>It would be a nice feature if Postmodern could help you with
478 defining your database schemas and, more importantly, updating
479 your databases when your code changes. It would theoretically not
480 be hard to build a function that compares a schema on the Lisp
481 side with the state of the database, and helps you to
482 interactively update your database. PostgreSQL has a quite
483 complete introspection system. Unfortunately it would be a lot of
484 work to implement this, since databases can contain so many
485 different types of entities (tables, views, indices, procedures,
486 constraints, sequences, etc.) which are all created, changed, and
487 dropped in different ways.
</p>
489 <h2 id=
"resources">Resources
</h2>
492 <li><a href=
"https://sites.google.com/site/sabraonthehill/postmodern-examples">A collection of Postmodern examples
</a></li>
493 <li><a href=
"http://www.postgresql.org/docs/current/static/index.html">The PostgreSQL manuals
</a></li>
494 <li><a href=
"http://www.postgresql.org/docs/current/static/protocol.html">The wire protocol Postmodern uses
</a></li>
495 <li><a href=
"http://clsql.b9.com/">CLSQL
</a></li>
496 <li><a href=
"https://github.com/filonenko-mikhail/cl-ewkb">Common Lisp Postgis library
</a></li>
497 <li><a href=
"http://common-lisp.net/project/local-time/">Local-time
</a></li>