1 #+OPTIONS: H:3 num:nil toc:2 \n:nil @:t ::t |:t ^:t -:t f:t *:t TeX:t LaTeX:t skip:nil d:(HIDE) tags:not-in-toc
2 #+STARTUP: align fold nodlcheck hidestars oddeven lognotestate
3 #+SEQ_TODO: TODO(t) INPROGRESS(i) WAITING(w@) | DONE(d) CANCELED(c@)
4 #+TAGS: Write(w) Update(u) Fix(f) Check(c)
5 #+TITLE: Advanced searching
7 #+EMAIL: mdl at imapmail dot org
12 [[file:index.org][{Back to Worg's index}]]
16 :CUSTOM_ID: introduction
19 Org-mode has many powerful built-in search functions. These tools
20 transform hierarchical org files into robust plain text "databases"
21 that can be queried in sophisticated ways. Outline headings in
22 Org-mode not only function as document sections or todo items; each
23 heading can also store an unlimited amount of text and various types
24 of metadata. And, of course, since Org-mode files are plain text, any
25 number of tools (grep, awk, perl, etc.) can be used to filter and
26 manipulate the data they contain.
28 The goal of this tutorial is to offer an introduction to the built-in
29 commands and syntax for querying Org-mode outlines. While these are
30 explained in various places in the Org-mode manual, this tutorial
31 attempts to provide an overview in one place. It is particularly aimed
32 at those who would like to use Org-mode as a note-taking and reference
33 management tool. Nonetheless, it should prove useful to anyone who
34 needs to locate specific information buried in an ever-growing
35 collection of Org-mode data.
37 * Outline nodes as "data containers"
39 :CUSTOM_ID: outline-nodes-as-data-containers
42 Before discussing specific search commands, it is worth taking a few
43 moments to consider the basic structure of an Org-mode entry.
45 Though outline entries can be nested within one another
46 hierarchically, each node is also a discrete container of data.
47 Indeed, a large variety of metadata---a todo keyword, tags,
48 timestamps, logging information, and properties (i.e., arbitrary data
49 pairs)---can be attached to each heading. Similarly each outline entry
50 can store an unlimited amount of text.
55 ,* TODO Buy clothes for wedding :wedding:important:errands:
56 , SCHEDULED: <2010-12-01 Wed>
58 , :estimated-cost: 100
60 , [2010-11-17 Wed 12:22]
62 , I need to look spiffy for the big day!
69 , Possible stores to visit:
71 , | Store | Location | Miles away |
72 , |-----------------+------------------------+------------|
73 , | The Suit King | 1000 E. Washington St. | 5.1 |
74 , | Mr. Haberdasher | 259 Western Rd. | 7.2 |
78 The sample entry above has the following metadata:
80 - todo keyword :: TODO
81 - scheduled timestamp :: 2010-12-01 Wed
82 - inactive timestamp :: 2010-11-17 Wed 12:22
83 - property :: estimated-cost \Rightarrow 100
84 - tags :: "wedding", "important", and "errands"
86 The entry also contains some text, including a checklist and a table.
88 Normally, an Org-mode file/outline contains several entries such as
89 the one above, nested hierarchically. Moreover, Org users typically
90 make their most important files available for easy searching by adding
91 them to their list of [[http://orgmode.org/manual/Agenda-files.html#Agenda-files][agenda files]], either selecting them one-by-one
92 with =C-c [= or by setting the variable =org-agenda-files=.
94 * The agenda as a search engine
96 :CUSTOM_ID: agenda-as-search-engine
99 For querying a collection of org files, Org-mode includes a powerful
100 built-in search-engine, [[http://orgmode.org/manual/Agenda-Views.html#Agenda-Views][the agenda]] (=C-c a=). As its name suggests,
101 the most common use of [[http://orgmode.org/manual/Agenda-Views.html#Agenda-Views][the agenda]] is to pull together, from all the
102 agenda files, a daily or weekly schedule or a list of todos. But the
103 agenda is also a powerful search engine that offers various ways tools
104 for querying both the metadata and the text of org-mode entries. In
105 fact, Org-mode's default agenda view (=C-c a a= or =org-agenda-list=)
106 is simply a search tool that gathers and displays all org-mode entries
107 with certain types of metadata---timestamps that fall within a given
110 Typing =C-c a= or =M-x org-agenda= brings up the agenda dispatcher, an
111 overview of Org-mode's various search tools:
113 : Press key for an agenda command: < Buffer, subtree/region restriction
114 : -------------------------------- > Remove restriction
115 : a Agenda for current week or day e Export agenda views
116 : t List of all TODO entries T Entries with special TODO kwd
117 : m Match a TAGS/PROP/TODO query M Like m, but only TODO entries
118 : L Timeline for current buffer # List stuck projects (!=configure)
119 : s Search for keywords C Configure custom agenda commands
120 : / Multi-occur ? Find :FLAGGED: entries
122 A quick perusal of [[http://orgmode.org/manual/Built_002din-agenda-views.html#Built_002din-agenda-views][the commands]] here reveals that one can query for a
123 wide variety of data. This tutorial will focus on three searches in
126 - =C-c a T= :: for todo keywords
127 - =C-c a m= :: for tags and properties
128 - =C-c a s= :: for full text searches
130 Now, let's look at the syntax for each of these search tools.
132 * Searching metadata (todos, tags, and properties)
134 :CUSTOM_ID: searching-metadata
136 ** Todo keyword searches
138 :CUSTOM_ID: todo-keyword-searches
141 The simplest type of metadata query in org-mode is [[http://orgmode.org/manual/Global-TODO-list.html#Global-TODO-list][=org-todo-list=]]
142 (invoked with =C-c a T=). This function prompts the user for a search
143 string and then retrieves a list of outline headings containing the
144 TODOs specified in the search string.[fn:lowercase-t]
146 Since each outline heading can contain only one TODO keyword, the
147 search syntax is quite simple, consisting either of a single keyword
148 or two or more keywords bound together by the boolean operator =|=
151 For instance, the following query...
155 ...retrieves all entries marked with a TODO keyword, whereas...
159 ...displays a list of all headlines containing either TODO or PROJECT
162 [fn:lowercase-t] Note that the lowercase variant of the command (=C-c
163 a t=) does not provide a search prompt, but simply pulls up all active
168 :CUSTOM_ID: tag-searches
171 Though the org-todo-list serves its purpose well, it is limited to
172 only one type of metadata. If you would like to search for other types
173 of metadata, or mix and match a search for todo keywords with, say, a
174 search for tags, org-mode offers a more powerful tool,
175 [[http://orgmode.org/manual/Matching-tags-and-properties.html#Matching-tags-and-properties][=org-tags-view=]], which is called with the following keys:
177 - =C-c a m= :: searches all headlines
178 - =C-c a M= :: searches only headlines with active todos
180 At its simplest, org-tags-view does exactly what it says: it queries
181 for headlines marked with particular combinations of tags. The syntax
182 for such searches follows a simple boolean logic:
186 - =+= :: include matches
187 - =-= :: exclude matches
189 Here are a few examples:
191 1. =+computer&+urgent=
193 ...will result in all items tagged both computer *and* urgent,
196 2. =+computer|+urgent=
198 ...will result in all items tagged *either* computer *or* urgent.
199 Meanwhile, the query...
201 3. =+computer&-urgent=
203 ...will display all items tagged computer *and not* urgent.
205 As you may have noticed, the syntax above can be a little verbose, so
206 org-mode offers convenient ways of shortening it. First, =-= and =+=
207 imply "and" if no boolean operator is stated, so example three above
208 could be rewritten simply as:
212 Second, inclusion of matches is implied if no =+= or =-= is present,
213 so example three could be further shortened to:
217 Example number two, meanwhile, could be shortened to:
223 :CUSTOM_ID: grouping-tags
226 There is no way (as yet) to express search grouping with parentheses.
227 The "and" operators (=&=, =+=, and =-=) always bind terms together
228 more strongly than "or" (=|=). For instance, the following search...
230 : computer|work+email
232 ...results in all headlines tagged either with "computer" or both
233 "work" and "email". An expression such as =(computer|work)&email= is
234 *not* supported at the moment.
236 There, are, however several other ways to achieve the grouping effect
239 1. Use a regular expression
241 To invoke the "grouping" logic of parentheses, you can construct a
244 : +{computer\|work}+email
246 Note: you can also use the special property =ALLTAGS= (which
247 queries the same data as a normal tags search) together with a
250 : ALLTAGS={computer\|work}+email
252 (In the next section we'll learn more about how property searches
253 and regular expressions are constructed.)
255 2. Use a slightly more verbose query as a substitute for the logic of
258 : computer&email|work&email
260 This search will match all headlines tagged either with "computer"
261 and "email" or with "work" and "email."
263 3. Use [[http://orgmode.org/manual/Agenda-commands.html#Agenda-commands][agenda filtering]].
265 Simply search for all headlines tagged with "computer" or "work"
266 and then use the agenda's tag filtering capabilities (=/=) to see
267 only those headlines among the results that have the tag "email."
271 :CUSTOM_ID: property-searches
274 Org-mode allows outline entries to contain any number of [[http://orgmode.org/manual/Properties-and-Columns.html#Properties-and-Columns][arbitrary
275 data pairs]], which are conveniently hidden within a folding PROPERTIES
279 ,* TODO Evensong's magisterial work on the Amazon :science:read:BIB:
280 , SCHEDULED: <2010-11-20 Sat>
281 , [2010-11-16 Tue 23:11]
283 , :BIB_AUTHOR: Walter Evensong
284 , :BIB_TITLE: Mysteries of the Amazon
286 , :BIB_PUBLISHER: Humbug University Press
289 , Lots of good stuff on Brazil.
292 Let's imagine a free software aficionado named Mr. Gnu has added a
293 number of similar bibliographical outline nodes to his org files and
294 that he would like to find all entries that contain "Walter Evensong"
295 in their =BIB_AUTHOR= field. He can construct such a search so by
296 calling =org-tags-view= and entering the desired key/value match:
299 : Match: BIB_AUTHOR="Walter Evensong"
301 Property searches can be mixed and matched with tag searches. If Mr.
302 Gnu would like to see all books by "Walter Evensong" with the tag
303 "read", he can simply join the two desired matches together with the
306 : BIB_AUTHOR="Walter Evensong"+read
308 Properties with numeric values can be queried with inequalities. If
309 Mr. Gnu would like to retrieve all books by the prolific Walter
310 Evensong that span over 1000 pages, he could enter the following:
312 : BIB_AUTHOR="Walter Evensong"+BIB_PAGES>1000
314 The comparison operators for searches are as follows:
316 : = (equal), > (greater than), <= (greater than or equal to),
317 : < (less than), <= (less than or equal to), <> (not equal)
319 What if Mr. Gnus would to like of find all books by Walter Evensong
320 *or* any books over 1000 pages?
322 : BIB_AUTHOR="Walter Evensong"|BIB_PAGES>1000
324 For his own clarity, Mr. Gnu can always insert "+" signs, though they
327 : +BIB_AUTHOR="Walter Evensong"|+BIB_PAGES>1000
329 It is important to note that the equal sign in the searches above
330 implies an exact match. If Mr. Gnu is searching for a string, such as
331 "Mysteries of the Amazon", the entire search query must match. Thus,
336 ...will not match the entry above.
338 How then can you search for partial matches? The answer is regular
339 expressions. Instead of surrounding your query with quotation marks
340 (which will necessitate a precise and complete match), you can instead
341 enfold it in brackets, which instructs Org-mode to treat the query as
342 a regular expression. Thus, the search...
346 ...will locate all entries that match contain the sequence "Amazon"
347 and pull them up in the agenda:
349 : Headlines with TAGS match: BIB_TITLE={Amazon}
350 : Press `C-u r' to search again with new search string
351 : org: TODO Evensong's magisterial work on the Amazon :science:read:BIB:
353 Mr. Gnu jots down the following rule in his growing org file collection:
356 ,* Tags/property search matching
357 , - For exact matches, use quotation marks.
358 , - For partial matches, use curly brackets.
361 [[http://www.gnu.org/s/emacs/manual/html_node/elisp/Regular-Expressions.html#Regular-Expressions][Regular expressions]] allow for more flexible searches. Let's say that
362 for some strange reason Mr. Gnu would like to find all books
363 containing either "Amazon" or "Amazing" in their titles. The following
364 regular expression search should do the trick:
366 : BIB_TITLE={Amaz\(on\|ing\)}
368 Let's break this expression down:
370 - =Amaz= :: This is the string shared by both words.
371 - =\(...\)= :: These parentheses create a grouping to set off the
372 alternative matches that follow "Amaz".
373 - =on\|ing= :: =\|= is the "or" expression. Since it is placed within
374 the parentheses, it means that a match must begin with
375 "Amaz" but can end *either* in "on" *or* "ing".
377 You may be wondering why the search query contains so many
378 backslashes. It is because Emacs' regular expression engine gives the
379 characters =(=, =)=, and =|= a special meaning only when they are
380 "escaped" (i.e., preceded by a backslash). Thus, Mr. Gnu had simply
381 typed =BIB_TITLE={Amaz(on|ing)}=, he would have instructed Org-mode to
382 match entries with the exact sequence =Amaz(on|ing)= (an unlikely
383 match, unless he has a large collection of literary theory from the
386 Here's a simpler example. If Mr. Gnu would like to find all entries
387 with either "Walter" or "Evensong" in the author field, he could type:
389 : BIB_TITLE={Walter\|Evensong}
391 If he would like to pull up all entries that have defined value for
392 the =BIB_TITLE= property, he can simply use a single dot to match any
397 ** Special Properties
399 :CUSTOM_ID: special-properties
402 In addition to any explicitly declared key/value property pairs, each
403 Org-mode entry also has a number of [[http://orgmode.org/manual/Special-properties.html#Special-properties][special (i.e., implicit)
404 properties]] that can be queried with =org-tags-view= (=C-c a m=). These
405 include, among other things, the entry's TODO state, tags (local and
406 inherited), category, priority, and timestamps (DEADLINE, SCHEDULED,
407 active, and inactive). See the [[#outline-nodes-as-data-containers][sample entry]] above for an illustration
408 of where these properties are typically found in an outline node.
410 To see all of the properties (both explicit and implicit) defined for
411 an Org-mode entry, place the following text in an org-mode entry and
412 evaluate it by typing C-x C-e after the closing parenthesis:
414 : (org-entry-properties nil)
416 Here's an example of how such "special properties" can be put to good
420 : Match: Effort>1+PRIORITY="A"+SCHEDULED<"<tomorrow>"+ALLTAGS={computer\|email}
422 This query finds all items with:
424 1. An estimated effort greater than one hour
426 3. A scheduled date "less than" tomorrow (i.e., today or earlier).
427 4. Either the tag "computer" or the tag "email"
428 - Note: the ALLTAGS property includes inherited tags, while the
429 TAGS property includes only local tags.
430 - This search is also a good example of how to achieve a [[#grouping-tags][grouping
431 logic without parentheses]] while querying tags.
433 Please [[http://orgmode.org/manual/Matching-tags-and-properties.html][consult the manual]] for a fuller explanation of the syntax of
436 *** Querying timestamps
438 :CUSTOM_ID: querying-timestamps
441 A few words should be said here about querying timestamps contained in
442 the following properties: =DEADLINE=, =SCHEDULED=, =TIMESTAMP= (the
443 first active timestamp in an entry), and =TIMESTAMP_IA= (the first
444 inactive timestamp in an entry).
446 The basic syntax for querying timestamps is a time string enclosed in
447 double quotes and angular brackets. E.g., the search...
450 : Match: +SCHEDULED="<2010-08-20 Sat>"
452 ...will find all items scheduled for Saturday, August 20, 2010
453 *without* a time of day specification. This last caveat is important
454 to note: if you have a timestamp with time of day information, such
459 , SCHEDULED: <2010-08-20-Sat 10:30>
462 ...the search above will not retrieve it. (This is not normally a
463 problem, since the daily/weekly agenda view provides a far superior
464 mechanism for viewing all timestamps that fall on a particular day.)
466 The true value of timestamp property queries lies in the use of
467 inequalities to capture a range of dates. To assist with this task,
468 Org-mode provides a number of convenient shortcuts:
470 - =<today>= and =<tomorrow>= :: timestamps for today and tomorrow
471 (without a time of day specification)
472 - =<now>= :: right now, including time of day
473 - e.g., =2010-11-20 Sat 12:42=
474 - =<-5d>=, =<-10w>=, =<+3m>=, =<+1y>= :: relative date indicators
475 - the shortcuts above indicate five days ago, ten weeks ago, three
476 months from now, and one year from now
478 To see all items SCHEDULED far in the future, say, more than a year
479 from now, you could type:
482 : Match: SCHEDULED>"<+1y>"
484 Here's another scenario. Imagine you use org-capture to take all your
485 notes and that you automatically stamp each notes with an inactive
486 timestamp. To find all notes you took in the past two weeks with the
487 tag "chimpanzees", you could perform the following search:
490 : Match: chimpanzees+TIMESTAMP_IA>="<-2w>"
492 *** Limit tags and properties searches by TODO state
494 :CUSTOM_ID: limiting-searches-to-todos
497 You can limit any of these tags/property searches to active todo
498 states simply by using =C-c a M= instead of =C-c a m=.
500 You can also, of course, limit the searches to a particular todo
501 keyword (say, NEXT) by adding...
505 ...to any of the searches above. But Org-mode also provides a
506 convenient (and more efficient) syntax for limiting searches to
507 particular TODO keywords. Simply add a =/= followed by a TODO search
508 in the form [[#todo-keyword-searches][we've already discussed]]. For instance, to limit the
509 chimpanzee search above to items marked DONE, you could type:
512 : Match: chimpanzees+TIMESTAMP_IA>="<-2w>"/DONE
514 As with normal todo searches, you can use or (=|=) to expand the
515 allowed matches. For instance, the query...
517 : chimpanzees+TIMESTAMP_IA>="<-2w>"/TODO|NEXT
519 ...will match against items marked either TODO or NEXT.
521 If you are matching only against active todos (i.e., things not marked
522 done), you can make your search more efficient by adding an exclamation
523 point. E.g., the following search...
525 : computer/!TODO|NEXT
527 ...will result in all items tagged "computer" and either a TODO or
528 NEXT keyword. The exclamation mark will speed up the search, because
529 org-mode will only query items that have an active todo keyword (as
530 defined either in the variable =org-todo-keywords= or in =#+TODO=
531 declarations at the top of an org file). For instance, if you had
532 placed the following line at the top of your org files...
534 : #+TODO: TODO NEXT STARTED WAITING | DONE CANCELED
536 ...an exclamation point limit the possible matches items marked TODO,
537 NEXT, STARTED, or WAITING.
539 You can use a a negative (=-=) to exclude TODO states. The search...
543 ...will result only in items marked TODO, NEXT, or STARTED.
545 Be careful to avoid using "and" logic when you query TODOs, since each
546 item, by definition can have only one TODO state. Take a look at the
547 following two searches:
549 : computer/!WAITING+TODO
551 : chimpanzees+TODO="TODO"+SCHEDULED<="<+1w>"+TODO="WAITING"
553 These searches will *never* return any positive results, since an org
554 entry cannot have both a TODO *and* a WAITING keyword.
556 * Searching the full text of entries
558 :CUSTOM_ID: searching-entry-text
562 :CUSTOM_ID: keyword-searches
565 Thus far, we have explored different ways to query the various types
566 of metadata attached to an org entry. But what if you would like to
567 search the entire text of your org entries?
569 The answer: call [[http://orgmode.org/manual/Search-view.html#Search-view][=org-search-view=]] with =C-c a s=. In the agenda
570 dispatcher, this appears as...
572 : s Search for keywords
574 Don't be fooled by the word "keywords," which some programs use as a
575 synonym for tags. Here, a keyword search scours the full text of org
578 Let's start with an example:
580 Desperately in need of typing practice (as if Emacs does not provide
581 enough keyboarding practice), our friend Mr. Gnu would like to locate
582 the following entry, which is buried somewhere in his agenda files:
585 ,* A sentence to test my keyboarding skills
587 ,The quick brown fox jumped over the lazy dog.
590 Mr. Gnu vaguely remembers that the entry contains the word "fox", so
591 he pecks at the keyboard to enter...
595 He is confronted with the prompt...
597 : [+-]Word/{Regexp} ...:
603 ...and receives an agenda buffer with the correct results:
606 : Press `[', `]' to add/sub word, `{', `}' to add/sub regexp, `C-u r' to edit
607 : typing: A sentence to test my keyboarding skills
609 Here, we should note that Org-mode's keyword searches are
610 *case-insensitive*, so "fox" will match any of the following: "fox",
613 Let's say, however, that Mr. Gnu's day job involves studying the
614 behavior of foxes, so he knows ahead of time that a simple search will
615 bring up hundreds of results. In addition, he recalls that the desired
616 entry also contains the word "dog". Thus, he enters the following:
619 : [+-]Word/{Regexp} ...: fox dog
621 Somewhat puzzlingly, Mr. Gnu's search yields no results. What went
624 Mr. Gnu consults [[http://orgmode.org/manual/Search-view.html#Search-view][the manual]] and finds that the default behavior of
625 =org-search-view= is to treat the entered query as a single string, so
626 when he typed =fox dog=, Org-mode looked quite literally for
627 =fox[whitespace]dog=.
629 Mr. Gnu further finds that to treat "dog" and "fox" as boolean
630 keywords that can be located anywhere in the entry, he needs to
631 precede each term with a =+=. (Technically, he only needs to precede
632 the first search term with =+= to initiate a boolean search, but he
633 decides to put =+= in front of both for the sake of clarity.) So he
637 : [+-]Word/{Regexp} ...: +fox +dog
639 ...and is overjoyed to retrieve the expected results.
641 Mr. Gnu makes a mental note: unless the first character of the search
642 query is a =+=, Org-mode will treat the entire query as a single
643 string. Thus, the query...
647 ...will prompt Org-mode to search for the single string "fox +dog".
648 (To change this behavior, please read the
649 [[#boolean-searches-by-default][section for "Google addicts" below]].)
651 Later, while at work, Mr. Gnu wants to find all entries on foxes that
652 do not contain the word dog, so he types...
655 : [+-]Word/{Regexp} ...: +fox -dog
657 If Mr. Gnu wants to incorporate a substring/phrase into a boolean
658 search (i.e., a query with a =+= at the beginning), he can use
663 At home again, while practicing typing, Mr. Gnu wants to find all
664 entries that contain either the word "keyboarding" or the word
665 "typing". Remember his lessons on tag searches, he tries the following
668 : +keyboarding|+typing
670 Alas, the search returns no results, because Mr. Gnu just instructed
671 Org-mode to look for the entire string "keyboarding|+typing." Reading
672 the manual, Mr. Gnu discovers that, unlike todo and tag searches,
673 keyword searches require separate terms to be separated by whitespace
674 (e.g., =+fox +dog=). In addition, Mr. Gnus realizes that keyword
675 searches have only two simple boolean expressions: =+= ("and") and =-=
676 ("and not"). There is no "or" symbol, such as =|=. What then should
677 Mr. Gnu do to find entries containing keyboarding *or* typing?
679 ** Full text search using regular expressions
681 :CUSTOM_ID: regexps-in-org-search-view
684 The solution to Mr. Gnu's puzzle is found in regular expressions.
685 Indeed, Mr. Gnu deduced as much by glancing at the org-search-view
688 : [+-]Word/{Regexp} ...:
690 As the prompt suggests, Mr. Gnu can search org-entries using Emacs'
691 powerful regular expression engine. To do so, he simply needs to
692 enclose the regular expression in brackets. So he types...
695 : [+-]Word/{Regexp} ...: +{keyboarding\|typing}
697 ...to find all entries that contain either "keyboarding" or "typing".
698 (Mr. Gnu could also have used parentheses to create a more compact
699 search query, such as =+{\(keyboard\|typ\)ing}=. Also, it is good to
700 recall here that =(=, =|=, and =)= only become special characters only
701 when escaped with a =\=.)
703 Regular expressions, Mr. Gnu finds, can be combined with words. The
706 : +{keyboarding\|typing} +fox
709 ...finds the "quick brown fox" entry above, while...
712 : +{keyboarding\|typing} -fox
714 ...excludes it, finding only those entries that contain either the
715 word "keyboarding" or "typing" and *not* the word dog.
717 Again, Org-mode's default behavior is to treat the entire query as a
718 single string unless it sees a =+= or a ={= at the beginning of the
719 line. So if Mr. Gnus types...
721 : dog +{keyboarding\|typing}
723 ...Org-mode will search for the entire substring "dog
724 +{keyboarding\|typing}". (If you don't like this behavior, please read
725 [[#boolean-searches-by-default][the section for "Google addicts" below]].)
727 *** Regular expression syntax
729 :CUSTOM_ID: regular-expression-syntax
732 The possibilities afforded by regular expressions are myriad. The
733 examples discussed here are relatively basic. For a thorough
734 introduction to regular expression syntax, please consult the [[http://www.gnu.org/s/emacs/manual/html_node/elisp/Syntax-of-Regexps.html#Syntax-of-Regexps][emacs
737 Let's look at a couple of examples:
739 Imagine you've entered a lot of contact entries with phone numbers in
740 the conventional U.S. format: 123-456-6789. To find all Org-mode
741 entries with such numbers, you could type:
744 : [+-]Word/{Regexp} ...: +{[0-9]\{3\}-[0-9]\{3\}-[0-9]\{4\}}
746 The square brackets here are special characters; they match any of
747 characters they enclose. For instance, =[abc]= matches either a or b
748 or c. In this particular case, the =[0-9]= matches any digit between 0
749 and 9. In addition, the escaped curly brackets (=\{...\}=) that
750 immediate follow the square brackets indicate how many times in a row
751 the character should occur. In this case, Org-mode will search for
752 the following sequence:
754 - exactly three digits
756 - exactly three digits
758 - exactly four digits
760 Instead of specifying the precise number of times a match such as
761 =[0-9]= must repeat, you can also use the following special
764 - =*= :: match any number of times (including none)
765 - =+= :: match at least once and possibly more
766 - =?= :: match either once or not at all
768 Now, imagine our friend Mr. Gnu is a new fan of Org-mode and has
769 jotted down a lot of notes on his favorite PIM. However, he have
770 entered the name Org-mode inconsistently, sometimes as "orgmode",
771 other times as "Org mode", and still other times as "Org-mode". He'd
772 like to find all his references to Org-mode, taking into account the
773 various spellings. Here's a simple query that will accomplish this:
777 Mr. Gnu just instructed Org-mode to search for any entry that contains
778 the character sequence "org", followed by a hyphen, a space, or no
779 character, followed by "mode". Since the search is case-insensitive,
780 it will match "org-mode", "org mode", or "orgmode".
782 ** Limiting full text searches
784 :CUSTOM_ID: limiting-full-text-searches
787 There are several convenient ways to refine and limit full text
790 First, if you find that a search produces too many results, you can
791 easily add a new word or regexp by typing any of the following in the
794 - =[= :: add a word (i.e., =+=)
795 - =]= :: exclude a word (i.e., =-=)
796 - ={= :: add a regexp (i.e., =+{}=)
797 - =}= :: exclude a regexp (i.e., =-{}=)
799 Let's say Mr. Gnu searches for the words Carsten *and* Dominik:
802 : [+-]Word/{Regexp} ...: +Carsten +Dominik
804 Since Mr. Gnu is an avid reader of the Org-mode mailing list and a
805 heavy user of org-capture, he discovers that he has hundreds of
806 entries that include Carsten's name. He wants to limit the search only
807 to entries with an inactive timestamp from November of 2010. So he
808 types =[= in the agenda buffer to add a new search term and receives the
811 : [+-]Word/{Regexp} ...: +Carsten +Dominik +
813 ...with the cursor conveniently located after the plus sign. He
814 completes the query to find inactive timestamps from November...
817 : [+-]Word/{Regexp} ...: +Carsten +Dominik +[2010-11-
819 ...and voilà , he retrieves a smaller subset of results.
821 If Mr. Gnu wants to find both active and inactive timestamps, he could
822 instead type ={= to add a regular expression:
824 : [+-]Word/{Regexp} ...: +Carsten +Dominik +{[\[<]2010-11-}
826 Similarly, if Mr. Gnu wants to guarantee the precision of his match,
827 he could use a detailed regular expression...
829 : +{\[2010-11-[0-9]\{2\}\s-[A-Za-z]\{3\}\(\s-[0-9]\{2\}:[0-9]\{2\}\)?\]}
831 But Mr. Gnu quickly decides that searching for the string "[2010-11-"
832 good enough for his purposes.
834 Org-mode also provides convenient syntax for limiting full text
837 1. If you place an asterisk at the beginning of your search, Org-mode
838 will search only headlines (and not entry text). E.g., to find all
839 entries with "emacs" in the headline, you could type:
842 : [+-]Word/{Regexp} ...: *+emacs
844 2. If you place an exclamation mark at the beginning of the query,
845 Org-mode will only pull up entries that are active todos:
849 (You can also limit your search to active todos by using a prefix
850 argument: =C-u C-c a s=.)
852 3. Finally, if you place a colon at the beginning of a query, the
853 boolean words you provide will only match entire words. Thus the
858 ...will match "emacs" but not "emacswiki".
860 You can mix and match these three limiting symbols, but they will only
861 work if they appear in the correct order: i.e., =*= -> =!= -> =:=. If
862 you type =:!+emacs=, your search will not retrieve any results.
864 ** Combining metadata and full text queries
866 :CUSTOM_ID: combining-metadata-and-full-text-queries
869 As an expert on tag and property searches, you might ask: is it
870 possible to combine metadata and full text searches? For instance, how
871 could Mr. Gnu find all entries with "Walter Evensong" in the
872 =BIB_AUTHOR= field, the todo keyword "DONE", and the word "Brazil" in
873 the full text of the entry?
875 - It is not possible to simply combine the syntax of metadata and
876 full text searches. Org-mode parses each query in fundamentally
879 - You can, however, easily accomplish "mixed" queries by using regular
880 expressions and =org-search-view=. In some instances,
881 =org-search-view= offers an easier and more efficient way of
882 querying metadata than the tags and property search.
884 The simplest way to think about Org-mode metadata is as different
885 types of markup patterns. Tags are enclosed in colons, todo keywords
886 directly follow the asterisks that mark outline headings, timestamps
887 are contained in brackets and have the pattern =YYYY-MM-DD DOW HH:MM=,
888 and so on. Thus, to query for particular types of metadata, one simply
889 has to construct regular expressions that match these patterns.
891 Back to Mr. Gnu, our mediocre typist and reader of very long books.
892 Today, he would like to find all entries in which:
894 - the =BIB_AUTHOR= is "Walter Evensong"
895 - the todo keyword is "DONE"
896 - the word "Brazil" appears in the full text.
898 First, he invokes =org-search-view=:
902 At the prompt, he adds a plus sign and the word "brazil":
904 : [+-]Word/{Regexp} ...: +brazil
906 He remembers that he must add the plus sign to instruct Org-mode to
907 treat this search as a boolean search. Otherwise it will simply look
908 for the entire string entered at the prompt.
910 Next he needs to search for the todo keyword "DONE". Since todo
911 keywords immediately follow the markup for outline headings, he can
912 simply add a regexp that matches an outline heading immediately
913 followed by the word DONE:
915 : [+-]Word/{Regexp} ...: +brazil +{^\*+\s-+DONE\s-}
917 This regexp begins with =^=, which forces a match at the beginning of
918 the line. It is followed by an asterisk, which needs to be escaped,
919 since an asterisk a special character in regular expressions. the =+=
920 after the asterisk instructs Org-mode to look for one or more
921 asterisks, while the =\s-+= indicates that at least one space follows
922 the asterisk(s). So Mr. Gnu is searching for at least one asterisk at
923 the beginning of the line followed by a space---the very definition of
924 an outline heading in Org-mode. And the keyword DONE followed by
925 whitespace completes the match. If Mr. Gnu would like to match more
926 than one todo keyword, say DONE or WAITING, he could use grouping:
927 =+{^\*+\s-+\(DONE\|WAITING\)\s-}=
929 Finally, Mr. Gnu finishes his query by searching for the property
930 =BIB_AUTHOR=. He recalls that a property line looks like this:
932 : :BIB_AUTHOR: Walter Evensong
934 With this in mind, he can easily construct a regexp to search for the
935 string =:BIB_AUTHOR:= followed by an arbitrary amount of whitespace
936 followed in turn by the string "Walter Evensong".
938 : [+-]Word/{Regexp} ...: +brazil +{^\*+\s-+DONE\s-} +{:BIB_AUTHOR:\s-+Walter Evensong}
940 Mr. Gnu is surprised at the speed with which Org-mode returns his
941 results. Indeed, he finds that regexp searches (especially those
942 querying properties) usually return their results more quickly than
943 property and tag searches. And he deduces the reason: whereas property
944 searches have to query each headline to determine whether a given
945 property contains a value, keyword searches simply scan each file for
946 matches and then return the appropriate headlines.
948 In many instances, of course, the DONE regexp above may be overkill.
949 Searching for the string "* DONE" will often do the trick. E.g.,
951 : [+-]Word/{Regexp} ...: +brazil +"* DONE"
953 Indeed, Mr. Gnu could probably also dispense with the =:BIB_AUTHOR:=
954 regexp above, simply typing...
956 : [+-]Word/{Regexp} ...: +brazil +"* DONE" +":BIB_AUTHOR: Walter Evensong"
958 Here's another example. Let's say Mr. Gnu would like to find all
959 active todos directly tagged "urgent" (i.e., not inherited) with the
960 word "wedding" somewhere in the entry text. The following keyword
961 search does the trick:
964 : [+-]Word/{Regexp} ...: !+wedding :urgent:
966 If Mr. Gnu wants to see either the tag "urgent" or the tag
967 "important", he could use a regular expression:
969 : !+wedding +{:\(urgent\|important\):}
971 The main limitation of such searches is that keyword searches know
972 nothing of outline tree inheritance. Thus, if Mr. Gnu is interested in
973 all entries that inherit the tag "urgent", he should always use
976 ** Searching additional files
978 :CUSTOM_ID: searching-additional-files
981 Often, the set of files one would like to search by keyword is larger
982 than one's set of active agenda files. For instance, one might archive
983 old projects in separate files so that they no longer contribute to
984 the agenda. Yet one would still like to search the reference material
985 in these projects by keyword/regexp.
987 The solution lies in the variable
988 =org-agenda-text-search-extra-files=. Adding a list of files to this
989 variable instructs =org-search-view= to query those files in addition
990 to the agenda files. Note that setting
991 =org-agenda-text-search-extra-files= has no effect on other types of
992 agenda commands, such as todo and tags/property searches.
994 ** Keyword searches for Google addicts
996 :CUSTOM_ID: boolean-searches-by-default
999 As noted before, =org-search-view= will treat a search query as a
1000 boolean expression only if it begins with either a =+= or a ={= (i.e.,
1001 a regular expression). Without these characters, Org-mode will treat
1002 the query as a single substring.
1004 This default syntax of =org-search-view= is thus different than the
1005 behavior of search engines such as Google, which treat searches as
1006 lazy boolean queries by default. If you type "emacs org-mode" into
1007 Google, it will not search for the literal string "emacs org-mode",
1008 but rather assume the space implies a boolean expression: "emacs and
1011 If you find yourself often forgetting to add an initial =+= to your
1012 =org-search-view= queries, you can make "lazy booleans" the default
1013 behavior by adding the following to your .emacs:
1016 (setq org-agenda-search-view-always-boolean t)
1019 Then you can happily type your lazy searches:
1022 : [+-]Word/{Regexp} ...: org mode Carsten :email:
1024 If you would like to include a substring or phrase in your search, you
1025 can do so by enclosing it in quotation marks. And if you want to
1026 exclude items or use regular expressions, you will, of course, still
1027 have to use a minus sign and curly brackets, respectively.
1029 * Searching org files line-by-line
1031 :CUSTOM_ID: line-by-line-search
1034 All the searches we have discussed thus far return their results as a
1035 list of org headlines in the agenda buffer. Sometimes, however, you
1036 might prefer to see each line in which a word or regular expression
1037 occurs. There are different ways to do this:
1041 :CUSTOM_ID: multi-occur
1044 Org-mode uses Emacs' multi-occur command to search for any lines in
1045 the agenda files containing a regular expression. Simply type =C-c a
1046 /= followed by a word or regular expression and you will be presented
1047 a buffer with all lines that match the query, with each line
1048 conveniently linked to its original location.
1050 ** External commands and scripts
1052 :CUSTOM_ID: grep-etc
1055 Emacs provides convenient interfaces to common Unix search commands,
1056 such as grep. Simply type =M-x grep= and complete the query (the
1057 working directory is usually that of the current buffer in Emacs).
1058 Using grep is especially convenient when you want quickly to search
1059 org files that are not in =org-agenda-files= or
1060 =org-agenda-text-search-extra-files=. And, of course, grep can be used
1063 Since org-mode files are plain text, you can use your favorite
1064 scripting language (perl, awk, python, etc.) to develop new and ever
1065 more creative ways to search and analyze them.
1069 :CUSTOM_ID: sparse-tree-view
1072 The commands we have examined so far typically search multiple files
1073 and display the resulting heading in a separate agenda buffer. But
1074 sometimes, you might want to search for various types of data within a
1075 single file, so as to see all the matching headlines and entries in
1078 The way to accomplish this is via a [[http://orgmode.org/manual/Sparse-trees.html#Sparse-trees][sparse tree view]] (=C-c /=), which
1079 collapses the outline in the current file, showing only the portions
1082 Calling =org-sparse-tree= with =C-c /= brings up a prompt with several
1085 : Sparse tree: [r]egexp [/]regexp [t]odo [T]odo-kwd [m]atch [p]roperty
1086 : [d]eadlines [b]efore-date [a]fter-date
1088 Some of these search, such as "todo" (=t=) and "deadlines" (=d=) are
1089 quite simple, showing all headlines in a buffer that contain an active
1090 todo keyword or a deadline, respectively. Others, such as "property"
1091 (=p=), prompt for a single key/value pair.
1093 One search that may be of particular interest is "match" (=m=). This
1094 query uses exactly the same syntax as =org-tags-view=, allowing us to
1095 use complex metadata searches to create sparse trees
1097 For instance, to highlight all active todos *without* a timestamp in
1098 the current buffer, you could type:
1101 : Match: -SCHEDULED={.}/!
1103 This instructs Org-mode to look for any active todo (=/!=) without a
1104 SCHEDULED timestamp.
1106 * Custom agenda commands
1108 :CUSTOM_ID: custom-agenda-commands
1111 If there are searches you perform again and again, you can easily save
1112 them by adding them to you [[http://orgmode.org/manual/Custom-agenda-views.html#Custom-agenda-views][custom agenda commands]].
1114 As we know, Mr. Gnu is an avid collector of very large books (which,
1115 of course, he manages in very long org files). Moreover, he often
1116 likes to peruse your inventory of books over 1,000 pages, querying his
1117 custom =BIB_PAGES= field. To save time and energy, Mr. Gnu could add a
1118 custom command such as the following to his =.emacs=:
1120 #+begin_src emacs-lisp
1121 (add-to-list 'org-agenda-custom-commands
1122 '("b" "Big books" tags "+BIB_PAGES>1000"))
1125 Note that "tags" here indicates =org-tags-view=. Thus, the query uses
1126 the tags/property search syntax.
1128 Mr. Gnu realizes he can save an even faster version of the search
1131 #+begin_src emacs-lisp
1132 (add-to-list 'org-agenda-custom-commands
1133 '("B" "Big books (fast)" search "{:BIB_PAGES:\\s-+[0-9]\\{4\\}}"))
1136 The symbol "search", as you might have guessed, instructs Org-mode to
1137 use =org-search-view=. And the saved search finds all items with
1138 =BIB_PAGES= property that contain four digits (i.e., > 1000 pages).
1140 You might notice that the search query here, compared with the one
1141 above, contains extra backslashes. That is because the backslash is a
1142 special character in emacs-lisp and thus needs to be escaped when
1143 placed in an =.emacs= file.
1145 If Mr. Gnu frequently need to perform the "urgent wedding tasks"
1146 search above, he could add a command such as the following:
1148 #+begin_src emacs-lisp
1149 (add-to-list 'org-agenda-custom-commands
1150 '("w" "Getting married next week!"
1151 search "!+wedding +{:\\(urgent\\|important\\):}"))
1154 Finally, one can use custom commands to run searches with different
1155 local settings. For instance, one can set up a custom agenda command
1156 to run a tags/property search on files other than the agenda files:
1158 #+begin_src emacs-lisp
1159 (add-to-list 'org-agenda-custom-commands
1160 '("r" "Reference material" tags ""
1161 ((org-agenda-files (file-expand-wildcards "~/ref/*.org")))))
1164 For a full introduction to custom agenda commands, please see [[file:org-custom-agenda-commands.org][this