reorienting what we want to do, based on current discussions with David and Steve.

[CommonLispStat.git] / README.org

Time-stamp: <2012-10-05 04:03:26 tony>

* Current Status: COMPLETELY BROKEN

  but we are rebuilding it.

* Fast Start

  You probably did  (preferred)

#+name: GitClone
#+begin_src shell
  git clone git://github.com/blindglobe/common-lisp-stat.git
#+end_src

  or (coming soon!) from within a Lisp instance, 

#+name: QuickLispLoad
#+begin_src lisp
  (ql:quickload :cls)
#+end_src

  At one point, I planned a pure git-delivery via cloning and
  submodules, but this proved to be a bit more complex than needed,
  thanks to the creation of quicklisp.  It's also a stupid idea if
  one plans to have users who are not hackers or developers, and
  eventually we want users.

  Despite quicklisp, there will need to be a version for delivering a
  system development-oriented CLS environment and this will consist of
  git repositories, possibly through submodules, but this (submodules)
  is for discussion.

  There are quite a few libraries that are needed, and right now we
  are working on simplifying the whole thing.   Once you get past
  this step, then you should:

  1. run a common lisp (SBCL, CMUCL, CLISP, CLOZURE-CL) starting in
     the current directory.  You will need ASDF at a minimum,
     QUICKLISP preferred.  And you should have QUICKLISP.

  2. (on Debian or similar systems: can use CLC (Common Lisp
     Controller) or SBCL approaches, i.e.  ~/.clc/systems or
     ~/.sbcl/systems should contain softlinks to the cls and other
     required ASDF files (i.e. cls.asd, cffi.asd, and lift.asd).

  There are example sessions and scripts for data analysis, some real,
  some proposed, in the file:examples/ directory.  Also see
  file:TODO.org for snippets of code that work or fail to work.

** Example Usage steps [2/6]

*** DONE Start and Load 
  
1. start your lisp
2. load CLS

#+BEGIN_SRC lisp
(ql:quickload :cls)
#+END_SRC

*** DONE Setup a place to work

In Common Lisp, you need to select and setup namespace to store data
and functions.  There is a scratch user-package, or sandbox, for
CLS, *cls-user* , which you can select via:

#+BEGIN_SRC lisp -n :tangle "readme-example.lisp"
(in-package :cls-user)
#+END_SRC

and this has some basic modules from CLS instantiated (dataframes,
probability calculus, numerical linear algebra, basic summaries
(numerical and visual displays).  

However, it can be better is to create a package to work in, which
pulls in only desired functionality:

#+BEGIN_SRC lisp +n :tangle "readme-example.lisp"
  (defpackage :my-package-user
    (:documentation "demo of how to put serious work should be placed in
      a similar package elsewhere for reproducibility.  This hints as to
      what needs to be done for a user- or analysis-package.")
    (:nicknames :my-clswork-user)
    (:use :common-lisp ; always needed for user playgrounds!
          :lisp-matrix ; we only need the packages that we need...
          :common-lisp-statistics
          :lisp-stat-data-examples) ;; this ensures access to a data package
    (:export summarize-data summarize-results this-data this-report)) 
  
  (in-package :my-clswork-user) ;; or :my-package-user
#+END_SRC

We need to pull in the packages with data or functions that we need;
just because the data/function is pulled in by another package, in
that package's namespace, does NOT mean it is available in this name
space.  However, the *common-lisp-statistics* package will ensure
that fundamental objects and functions are always available. 


*** TODO Get to work [0/3]

**** TODO Pull in or create data

**** TODO Summarize results

**** TODO Save work and results for knowledge building and reuse 

One can build a package, or save an image (CL implementation
dependent) or...
  
*** Inform  moi of problems or successes

    mailto:blindglobe@gmail.com if there is anything wrong, or
    even if something happens to work.

    Current beliefs:
    - SBCL is target platform.   CCL and CMUCL should be similar.
    - CLISP is finicky regarding the problems that we have with CFFI
      conversation.  In particular that we can not really do typing
      that we need to take care of.  I think this is my (Tony's)
      problem, not someone elses, and specifically, not CLISP's
    - Need to test ECL.

* History

   See files in file:Doc/  for history, design considerations, and
   random, sometimes false and misleading, musings.

* Working on your own modifications

#+begin_src shell
   git clone git://repo.or.cz/CommonLispStat.git 
   cd CommonLispStat
#   git submodules init
#   git submodules update
#+end_src

   will pull the whole repository, and create a "master" branch to
   work on.  If you are making edits, which I'd like, you don't want
   to use the master branch, but more to use a topic-centric branch,
   so you might:

#+begin_src shell
    git checkout -b myTopicBranch
#+end_src

and then work on myTopicBranch, pulling back to the master branch when
needed by

#+begin_src shell
    git checkout master
    git pull . myTopicBranch
#+end_src

(or
#+begin_src shell
    git rebase myTopicBranch
#+end_src
)

of course, perhaps you want to contribute to the mob branch.   For
that, after cloning the repository as above, you would:

#+begin_src shell
    git checkout -b mob remotes/origin/mob
#+end_src

(work, work, work... through a cycle of

#+begin_src shell
         <edit>
         git add <files just edited>
         git commit -m "what I just did"
#+end_src

 ad-nauseum.  When ready to commit, then just:

#+begin_src shell
     git push git+ssh://mob@repo.or.cz/srv/git/CommonLispStat.git mob:mob
#+end_src

)

and it'll be put on the mob branch, as a proposal for merging.

Another approach would be to pull from the topic branch into the mob
branch before uploading.   Will work on a formal example soon.

(the basic principle is that instead of the edit cycle on mob, do
something like:

#+begin_src shell
  git checkout mob
  git pull . myTopicBranch   
  git push git+ssh://mob@repo.or.cz/srv/git/CommonLispStat.git mob:mob
#+end_src

)

Alternatively, one can work on the github repositories as well.  They
are a bit differently organized, and require one to get a github
account and work from there.  In that case, you'd need to D/L the
libraries. 

That gets a bit tricky, but see ./bin/GetRepos.sh for an example. 

* Documentation and examples

  I've started putting examples of use in function documentation.  If
  you are a lisp'er, you'll find this pendantic and insulting.  Many
  of the uses are trivial.  However, this has been tested out on a
  number of research statisticians (the primary user audience) and
  found useful.

  Still need to write the (run-doc-ex 'function-name) function, which
  would print out the example and run it live.  Hopefully with the
  same results.  I've used XML markup for this, but for no particular
  reason, we could have used SEXPs as well.  This is currently done by
  using an <example> tag set, as in

#+srcname: 
#+begin_src xml
  <example>
  (progn
    (example-code-for-function))
  </example>
#+end_src

* Footnotes

[fn:1] I´m not including instructions for Emacs or git, as the former
is dealt with other places and the latter was required for you to get
this.  Since disk space is cheap, I´m intentionally forcing git to be
part of this system.  Sorry if you hate it.  Org-mode, org-babel, and
org-babel-lisp, and hypo are useful for making this file a literate
and interactively executable piece of work.