2 # Script to perform searching
3 # For documentation try 'perldoc /path/to/search'
5 # Copyright 2006 LibLime
7 # This file is part of Koha
9 # Koha is free software; you can redistribute it and/or modify it under the
10 # terms of the GNU General Public License as published by the Free Software
11 # Foundation; either version 2 of the License, or (at your option) any later
14 # Koha is distributed in the hope that it will be useful, but WITHOUT ANY
15 # WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
16 # A PARTICULAR PURPOSE. See the GNU General Public License for more details.
18 # You should have received a copy of the GNU General Public License along
19 # with Koha; if not, write to the Free Software Foundation, Inc.,
20 # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
24 search - a search script for finding records in a Koha system (Version 3)
28 This script utilizes a new search API for Koha 3. It is designed to be
29 simple to use and configure, yet capable of performing feats like stemming,
30 field weighting, relevance ranking, support for multiple query language
31 formats (CCL, CQL, PQF), full support for the bib1 attribute set, extended
32 attribute sets defined in Zebra profiles, access to the full range of Z39.50
33 and SRU query options, federated searches on Z39.50/SRU targets, etc.
35 The API as represented in this script is mostly sound, even if the individual
36 functions in Search.pm and Koha.pm need to be cleaned up. Of course, you are
39 I will attempt to describe what is happening at each part of this script.
40 -- Joshua Ferraro <jmf AT liblime DOT com>
44 This script performs two functions:
48 =item 1. interacts with Koha to retrieve and display the results of a search
50 =item 2. loads the advanced search page
54 These two functions share many of the same variables and modules, so the first
55 task is to load what they have in common and determine which template to use.
56 Once determined, proceed to only load the variables and procedures necessary
59 =head2 LOADING ADVANCED SEARCH PAGE
61 This is fairly straightforward, and I won't go into detail ;-)
63 =head2 PERFORMING A SEARCH
65 If we're performing a search, this script performs three primary
70 =item 1. builds query strings (yes, plural)
72 =item 2. perform the search and return the results array
74 =item 3. build the HTML for output to the template
78 There are several additional secondary functions performed that I will
81 =head3 1. Building Query Strings
83 There are several types of queries needed in the process of search and retrieve:
87 =item 1 $query - the fully-built query passed to zebra
89 This is the most complex query that needs to be built. The original design goal
90 was to use a custom CCL2PQF query parser to translate an incoming CCL query into
91 a multi-leaf query to pass to Zebra. It needs to be multi-leaf to allow field
92 weighting, koha-specific relevance ranking, and stemming. When I have a chance
93 I'll try to flesh out this section to better explain.
95 This query incorporates query profiles that aren't compatible with most non-Zebra
96 Z39.50 targets to acomplish the field weighting and relevance ranking.
98 =item 2 $simple_query - a simple query that doesn't contain the field weighting,
99 stemming, etc., suitable to pass off to other search targets
101 This query is just the user's query expressed in CCL CQL, or PQF for passing to a
102 non-zebra Z39.50 target (one that doesn't support the extended profile that Zebra does).
104 =item 3 $query_cgi - passed to the template / saved for future refinements of
107 This is a simple string that completely expresses the query as a CGI string that
108 can be used for future refinements of the query or as a part of a history feature.
110 =item 4 $query_desc - Human search description - what the user sees in search
113 This is a simple string that is human readable. It will contain '=', ',', etc.
117 =head3 2. Perform the Search
119 This section takes the query strings and performs searches on the named servers,
120 including the Koha Zebra server, stores the results in a deeply nested object,
121 builds 'faceted results', and returns these objects.
125 The final major section of this script takes the objects collected thusfar and
126 builds the HTML for output to the template and user.
128 =head3 Additional Notes
134 use strict
; # always use
135 #use warnings; FIXME - Bug 2505
137 ## STEP 1. Load things that are used in both search page and
138 # results page and decide which template to load, operations
144 use C4
::Auth
qw(:DEFAULT get_session);
146 use C4
::Languages
qw(getAllLanguages);
148 use C4
::VirtualShelves
qw(GetRecentShelves);
149 use POSIX
qw(ceil floor);
150 use C4
::Branch
; # GetBranches
152 # create a new CGI object
153 # FIXME: no_undef_params needs to be tested
154 use CGI
qw('-no_undef_params');
157 my ($template,$borrowernumber,$cookie);
159 # decide which template to use
162 my @params = $cgi->param("limit");
163 if ((@params>=1) || ($cgi->param("q")) || ($cgi->param('multibranchlimit')) || ($cgi->param('limit-yr')) ) {
164 $template_name = 'catalogue/results.tmpl';
167 $template_name = 'catalogue/advsearch.tmpl';
168 $template_type = 'advsearch';
171 ($template, $borrowernumber, $cookie) = get_template_and_user
({
172 template_name
=> $template_name,
175 authnotrequired
=> 0,
176 flagsrequired
=> { catalogue
=> 1 },
179 if (C4
::Context
->preference("marcflavour") eq "UNIMARC" ) {
180 $template->param('UNIMARC' => 1);
184 # Deprecated, but preserved because it's interesting :-)
185 # The same thing can be accomplished with mod_rewrite in
189 #my $uri = $cgi->url(-base => 1);
190 #my $relative_url = $cgi->url(-relative=>1);
191 #$uri.="/".$relative_url."?";
193 #my @cgi_params_list = $cgi->param();
194 #my $url_params = $cgi->Vars;
196 #for my $each_param_set (@cgi_params_list) {
197 # $uri.= join "", map "\&$each_param_set=".$_, split("\0",$url_params->{$each_param_set}) if $url_params->{$each_param_set};
199 #warn "New URI:$uri";
200 # Only re-write a URI if there are params or if it already hasn't been re-written
201 #unless (($cgi->param('r')) || (!$cgi->param()) ) {
202 # print $cgi->redirect( -uri=>$uri."&r=1",
203 # -cookie => $cookie);
208 my $branches = GetBranches
();
211 # we need to know the borrower branch code to set a default branch
212 my $borrowerbranchcode = C4
::Context
->userenv->{'branch'};
214 for my $branch_hash (sort { $branches->{$a}->{branchname
} cmp $branches->{$b}->{branchname
} } keys %$branches) {
215 # if independantbranches is activated, set the default branch to the borrower branch
216 my $selected = (C4
::Context
->preference("independantbranches") and ($borrowerbranchcode eq $branch_hash)) ?
1 : undef;
217 push @branch_loop, {value
=> "$branch_hash" , branchname
=> $branches->{$branch_hash}->{'branchname'}, selected
=> $selected};
220 my $categories = GetBranchCategories
(undef,'searchdomain');
222 $template->param(branchloop
=> \
@branch_loop, searchdomainloop
=> $categories);
224 # load the Type stuff
225 # load the Type stuff
226 my $itemtypes = GetItemTypes
;
227 # the index parameter is different for item-level itemtypes
228 my $itype_or_itemtype = (C4
::Context
->preference("item-level_itypes"))?
'itype':'itemtype';
232 my $advanced_search_types = C4
::Context
->preference("AdvancedSearchTypes");
234 if (!$advanced_search_types or $advanced_search_types eq 'itemtypes') { foreach my $thisitemtype ( sort {$itemtypes->{$a}->{'description'} cmp $itemtypes->{$b}->{'description'} } keys %$itemtypes ) {
235 my %row =( number
=>$cnt++,
236 ccl
=> $itype_or_itemtype,
237 code
=> $thisitemtype,
238 selected
=> $selected,
239 description
=> $itemtypes->{$thisitemtype}->{'description'},
241 imageurl
=> getitemtypeimagelocation
( 'intranet', $itemtypes->{$thisitemtype}->{'imageurl'} ),
243 $selected = 0 if ($selected) ;
244 push @itemtypesloop, \
%row;
246 $template->param(itemtypeloop
=> \
@itemtypesloop);
248 my $advsearchtypes = GetAuthorisedValues
($advanced_search_types);
249 for my $thisitemtype (sort {$a->{'lib'} cmp $b->{'lib'}} @
$advsearchtypes) {
252 ccl
=> $advanced_search_types,
253 code
=> $thisitemtype->{authorised_value
},
254 selected
=> $selected,
255 description
=> $thisitemtype->{'lib'},
257 imageurl
=> getitemtypeimagelocation
( 'intranet', $thisitemtype->{'imageurl'} ),
259 push @itemtypesloop, \
%row;
261 $template->param(itemtypeloop
=> \
@itemtypesloop);
264 # The following should only be loaded if we're bringing up the advanced search template
265 if ( $template_type eq 'advsearch' ) {
267 # load the servers (used for searching -- to do federated searching, etc.)
268 my $primary_servers_loop;# = displayPrimaryServers();
269 $template->param(outer_servers_loop
=> $primary_servers_loop,);
271 my $secondary_servers_loop;
272 $template->param(outer_sup_servers_loop
=> $secondary_servers_loop,);
274 # set the default sorting
275 my $default_sort_by = C4
::Context
->preference('defaultSortField')."_".C4
::Context
->preference('defaultSortOrder')
276 if (C4
::Context
->preference('OPACdefaultSortField') && C4
::Context
->preference('OPACdefaultSortOrder'));
277 $template->param($default_sort_by => 1);
279 # determine what to display next to the search boxes (ie, boolean option
280 # shouldn't appear on the first one, scan indexes should, adding a new
281 # box should only appear on the last, etc.
282 my @search_boxes_array;
283 my $search_boxes_count = C4
::Context
->preference("OPACAdvSearchInputCount") || 3; # FIXME: using OPAC sysprefs?
284 # FIXME: all this junk can be done in TMPL using __first__ and __last__
285 for (my $i=1;$i<=$search_boxes_count;$i++) {
286 # if it's the first one, don't display boolean option, but show scan indexes
288 push @search_boxes_array, {scan_index
=> 1};
290 # if it's the last one, show the 'add field' box
291 elsif ($i==$search_boxes_count) {
292 push @search_boxes_array,
299 push @search_boxes_array,
306 $template->param(uc(C4
::Context
->preference("marcflavour")) => 1,
307 search_boxes_loop
=> \
@search_boxes_array);
309 # load the language limits (for search)
310 my $languages_limit_loop = getAllLanguages
();
311 $template->param(search_languages_loop
=> $languages_limit_loop,);
313 # use the global setting by default
314 if ( C4
::Context
->preference("expandedSearchOption") == 1) {
315 $template->param( expanded_options
=> C4
::Context
->preference("expandedSearchOption") );
317 # but let the user override it
318 if ( ($cgi->param('expanded_options') == 0) || ($cgi->param('expanded_options') == 1 ) ) {
319 $template->param( expanded_options
=> $cgi->param('expanded_options'));
322 $template->param(virtualshelves
=> C4
::Context
->preference("virtualshelves"));
324 output_html_with_http_headers
$cgi, $cookie, $template->output;
328 ### OK, if we're this far, we're performing a search, not just loading the advanced search page
330 # Fetch the paramater list as a hash in scalar context:
331 # * returns paramater list as tied hash ref
332 # * we can edit the values by changing the key
333 # * multivalued CGI paramaters are returned as a packaged string separated by "\0" (null)
334 my $params = $cgi->Vars;
336 # Params that can have more than one value
337 # sort by is used to sort the query
338 # in theory can have more than one but generally there's just one
340 my $default_sort_by = C4
::Context
->preference('defaultSortField')."_".C4
::Context
->preference('defaultSortOrder')
341 if (C4
::Context
->preference('defaultSortField') && C4
::Context
->preference('defaultSortOrder'));
343 @sort_by = split("\0",$params->{'sort_by'}) if $params->{'sort_by'};
344 $sort_by[0] = $default_sort_by unless $sort_by[0];
345 foreach my $sort (@sort_by) {
346 $template->param($sort => 1);
348 $template->param('sort_by' => $sort_by[0]);
350 # Use the servers defined, or just search our local catalog(default)
352 @servers = split("\0",$params->{'server'}) if $params->{'server'};
354 #FIXME: this should be handled using Context.pm
355 @servers = ("biblioserver");
356 # @servers = C4::Context->config("biblioserver");
358 # operators include boolean and proximity operators and are used
359 # to evaluate multiple operands
361 @operators = split("\0",$params->{'op'}) if $params->{'op'};
363 # indexes are query qualifiers, like 'title', 'author', etc. They
364 # can be single or multiple parameters separated by comma: kw,right-Truncation
366 @indexes = split("\0",$params->{'idx'});
368 # if a simple index (only one) display the index used in the top search box
369 if ($indexes[0] && !$indexes[1]) {
370 $template->param("ms_".$indexes[0] => 1);}
373 # an operand can be a single term, a phrase, or a complete ccl query
375 @operands = split("\0",$params->{'q'}) if $params->{'q'};
377 # limits are use to limit to results to a pre-defined category such as branch or language
379 @limits = split("\0",$params->{'limit'}) if $params->{'limit'};
381 if($params->{'multibranchlimit'}) {
382 push @limits, join(" or ", map { "branch: $_ "} @
{GetBranchesInCategory
($params->{'multibranchlimit'})}) ;
386 foreach my $limit(@limits) {
387 if ($limit =~/available/) {
391 $template->param(available
=> $available);
393 # append year limits if they exist
396 if ($params->{'limit-yr'}) {
397 if ($params->{'limit-yr'} =~ /\d{4}-\d{4}/) {
398 my ($yr1,$yr2) = split(/-/, $params->{'limit-yr'});
399 $limit_yr = "yr,st-numeric,ge=$yr1 and yr,st-numeric,le=$yr2";
400 $limit_yr_value = "$yr1-$yr2";
402 elsif ($params->{'limit-yr'} =~ /\d{4}/) {
403 $limit_yr = "yr,st-numeric=$params->{'limit-yr'}";
404 $limit_yr_value = $params->{'limit-yr'};
406 push @limits,$limit_yr;
407 #FIXME: Should return a error to the user, incorect date format specified
410 # convert indexes and operands to corresponding parameter names for the z3950 search
411 # $ %z3950p will be a hash ref if the indexes are present (advacned search), otherwise undef
413 my $indexes2z3950 = {
414 kw
=>'title', au
=>'author', 'au,phr'=>'author', nb
=>'isbn', ns
=>'issn',
415 'lcn,phr'=>'dewey', su
=>'subject', 'su,phr'=>'subject',
416 ti
=>'title', 'ti,phr'=>'title', se
=>'title'
418 for (my $ii = 0; $ii < @operands; ++$ii)
420 my $name = $indexes2z3950->{$indexes[$ii]};
421 if (defined $name && defined $operands[$ii])
424 $z3950par->{$name} = $operands[$ii] if !exists $z3950par->{$name};
429 # Params that can only have one value
430 my $scan = $params->{'scan'};
431 my $count = C4
::Context
->preference('numSearchResults') || 20;
432 my $results_per_page = $params->{'count'} || $count;
433 my $offset = $params->{'offset'} || 0;
434 my $page = $cgi->param('page') || 1;
435 #my $offset = ($page-1)*$results_per_page;
437 my $expanded_facet = $params->{'expand'};
439 # Define some global variables
440 my ( $error,$query,$simple_query,$query_cgi,$query_desc,$limit,$limit_cgi,$limit_desc,$stopwords_removed,$query_type);
444 ## I. BUILD THE QUERY
445 my $lang = C4
::Output
::getlanguagecookie
($cgi);
446 ( $error,$query,$simple_query,$query_cgi,$query_desc,$limit,$limit_cgi,$limit_desc,$stopwords_removed,$query_type) = buildQuery
(\
@operators,\
@operands,\
@indexes,\
@limits,\
@sort_by,$scan,$lang);
448 ## parse the query_cgi string and put it into a form suitable for <input>s
450 my $scan_index_to_use;
452 for my $this_cgi ( split('&',$query_cgi) ) {
453 next unless $this_cgi;
454 $this_cgi =~ m/(.*=)(.*)/;
456 my $input_value = $2;
457 $input_name =~ s/=$//;
458 push @query_inputs, { input_name
=> $input_name, input_value
=> $input_value };
459 if ($input_name eq 'idx') {
460 $scan_index_to_use = $input_value; # unless $scan_index_to_use;
463 $template->param ( QUERY_INPUTS
=> \
@query_inputs,
464 scan_index_to_use
=> $scan_index_to_use );
466 ## parse the limit_cgi string and put it into a form suitable for <input>s
468 for my $this_cgi ( split('&',$limit_cgi) ) {
469 next unless $this_cgi;
470 # handle special case limit-yr
471 if ($this_cgi =~ /yr,st-numeric/) {
472 push @limit_inputs, { input_name
=> 'limit-yr', input_value
=> $limit_yr_value };
475 $this_cgi =~ m/(.*=)(.*)/;
477 my $input_value = $2;
478 $input_name =~ s/=$//;
479 push @limit_inputs, { input_name
=> $input_name, input_value
=> $input_value };
481 $template->param ( LIMIT_INPUTS
=> \
@limit_inputs );
483 ## II. DO THE SEARCH AND GET THE RESULTS
484 my $total; # the total results for the whole set
485 my $facets; # this object stores the faceted results that display on the left-hand of the results page
489 if (C4
::Context
->preference('NoZebra')) {
490 $query=~s/yr(:|=)\s*([\d]{1,4})-([\d]{1,4})/(yr>=$2 and yr<=$3)/g;
491 $simple_query=~s/yr\s*(:|=)([\d]{1,4})-([\d]{1,4})/(yr>=$2 and yr<=$3)/g;
494 ($error, $results_hashref, $facets) = NZgetRecords
($query,$simple_query,\
@sort_by,\
@servers,$results_per_page,$offset,$expanded_facet,$branches,$query_type,$scan);
498 ($error, $results_hashref, $facets) = getRecords
($query,$simple_query,\
@sort_by,\
@servers,$results_per_page,$offset,$expanded_facet,$branches,$query_type,$scan);
502 $template->param(query_error
=> $error.$@
);
503 output_html_with_http_headers
$cgi, $cookie, $template->output;
507 # At this point, each server has given us a result set
508 # now we build that set for template display
509 my @sup_results_array;
510 for (my $i=0;$i<@servers;$i++) {
511 my $server = $servers[$i];
512 if ($server =~/biblioserver/) { # this is the local bibliographic server
513 $hits = $results_hashref->{$server}->{"hits"};
514 my $page = $cgi->param('page') || 0;
515 my @newresults = searchResults
( $query_desc,$hits,$results_per_page,$offset,$scan,@
{$results_hashref->{$server}->{"RECORDS"}});
516 $total = $total + $results_hashref->{$server}->{"hits"};
517 ## If there's just one result, redirect to the detail page
519 my $biblionumber = $newresults[0]->{biblionumber
};
520 my $defaultview = C4
::Context
->preference('IntranetBiblioDefaultView');
521 my $views = { C4
::Search
::enabled_staff_search_views
};
522 if ($defaultview eq 'isbd' && $views->{can_view_ISBD
}) {
523 print $cgi->redirect("/cgi-bin/koha/catalogue/ISBDdetail.pl?biblionumber=$biblionumber");
524 } elsif ($defaultview eq 'marc' && $views->{can_view_MARC
}) {
525 print $cgi->redirect("/cgi-bin/koha/catalogue/MARCdetail.pl?biblionumber=$biblionumber");
526 } elsif ($defaultview eq 'labeled_marc' && $views->{can_view_labeledMARC
}) {
527 print $cgi->redirect("/cgi-bin/koha/catalogue/labeledMARCdetail.pl?biblionumber=$biblionumber");
529 print $cgi->redirect("/cgi-bin/koha/catalogue/detail.pl?biblionumber=$biblionumber");
536 $template->param(total
=> $hits);
537 my $limit_cgi_not_availablity = $limit_cgi;
538 $limit_cgi_not_availablity =~ s/&limit=available//g;
539 $template->param(limit_cgi_not_availablity
=> $limit_cgi_not_availablity);
540 $template->param(limit_cgi
=> $limit_cgi);
541 $template->param(query_cgi
=> $query_cgi);
542 $template->param(query_desc
=> $query_desc);
543 $template->param(limit_desc
=> $limit_desc);
544 $template->param (z3950_search_params
=> C4
::Search
::z3950_search_args
($query_desc));
545 if ($query_desc || $limit_desc) {
546 $template->param(searchdesc
=> 1);
548 $template->param(stopwords_removed
=> "@$stopwords_removed") if $stopwords_removed;
549 $template->param(results_per_page
=> $results_per_page);
550 $template->param(SEARCH_RESULTS
=> \
@newresults);
552 ## FIXME: add a global function for this, it's better than the current global one
553 ## Build the page numbers on the bottom of the page
555 # total number of pages there will be
556 my $pages = ceil
($hits / $results_per_page);
557 # default page number
558 my $current_page_number = 1;
559 $current_page_number = ($offset / $results_per_page + 1) if $offset;
560 my $previous_page_offset = $offset - $results_per_page unless ($offset - $results_per_page <0);
561 my $next_page_offset = $offset + $results_per_page;
562 # If we're within the first 10 pages, keep it simple
563 #warn "current page:".$current_page_number;
564 if ($current_page_number < 10) {
565 # just show the first 10 pages
566 # Loop through the pages
567 my $pages_to_show = 10;
568 $pages_to_show = $pages if $pages<10;
569 for (my $i=1; $i<=$pages_to_show;$i++) {
570 # the offset for this page
571 my $this_offset = (($i*$results_per_page)-$results_per_page);
572 # the page number for this page
573 my $this_page_number = $i;
574 # it should only be highlighted if it's the current page
575 my $highlight = 1 if ($this_page_number == $current_page_number);
576 # put it in the array
577 push @page_numbers, { offset
=> $this_offset, pg
=> $this_page_number, highlight
=> $highlight, sort_by
=> join " ",@sort_by };
583 # now, show twenty pages, with the current one smack in the middle
585 for (my $i=$current_page_number; $i<=($current_page_number + 20 );$i++) {
586 my $this_offset = ((($i-9)*$results_per_page)-$results_per_page);
587 my $this_page_number = $i-9;
588 my $highlight = 1 if ($this_page_number == $current_page_number);
589 if ($this_page_number <= $pages) {
590 push @page_numbers, { offset
=> $this_offset, pg
=> $this_page_number, highlight
=> $highlight, sort_by
=> join " ",@sort_by };
594 # FIXME: no previous_page_offset when pages < 2
595 $template->param( PAGE_NUMBERS
=> \
@page_numbers,
596 previous_page_offset
=> $previous_page_offset) unless $pages < 2;
597 $template->param( next_page_offset
=> $next_page_offset) unless $pages eq $current_page_number;
603 $template->param(searchdesc
=> 1,query_desc
=> $query_desc,limit_desc
=> $limit_desc);
604 $template->param (z3950_search_params
=> C4
::Search
::z3950_search_args
($z3950par || $query_desc));
607 } # end of the if local
609 # asynchronously search the authority server
610 elsif ($server =~/authorityserver/) { # this is the local authority server
611 my @inner_sup_results_array;
612 for my $sup_record ( @
{$results_hashref->{$server}->{"RECORDS"}} ) {
613 my $marc_record_object = MARC
::Record
->new_from_usmarc($sup_record);
614 # warn "Authority Found: ".$marc_record_object->as_formatted();
615 push @inner_sup_results_array, {
616 'title' => $marc_record_object->field(100)->subfield('a'),
617 'link' => "&idx=an&q=".$marc_record_object->field('001')->as_string(),
620 push @sup_results_array, { servername
=> $server,
621 inner_sup_results_loop
=> \
@inner_sup_results_array} if @inner_sup_results_array;
623 # FIXME: can add support for other targets as needed here
624 $template->param( outer_sup_results_loop
=> \
@sup_results_array);
625 } #/end of the for loop
626 #$template->param(FEDERATED_RESULTS => \@results_array);
630 #classlist => $classlist,
633 facets_loop
=> $facets,
635 search_error
=> $error,
638 if ($query_desc || $limit_desc) {
639 $template->param(searchdesc
=> 1);
642 # VI. BUILD THE TEMPLATE
644 # Build drop-down list for 'Add To:' menu...
646 my $row_count = 10; # FIXME:This probably should be a syspref
647 my ($pubshelves, $total) = GetRecentShelves
(2, $row_count, undef);
648 my ($barshelves, $total) = GetRecentShelves
(1, $row_count, $borrowernumber);
650 my @pubshelves = @
{$pubshelves};
651 my @barshelves = @
{$barshelves};
654 $template->param( addpubshelves
=> scalar (@pubshelves));
655 $template->param( addpubshelvesloop
=> @pubshelves);
659 $template->param( addbarshelves
=> scalar (@barshelves));
660 $template->param( addbarshelvesloop
=> @barshelves);
665 output_html_with_http_headers
$cgi, $cookie, $template->output;