1 #+Title: Neural Network
3 #+STYLE: <link rel="stylesheet" href="/~eschulte/classes/stylesheet.css" type="text/css">
5 #+Options: toc:nil num:nil
8 A domain specific language for expressing neural networks using
9 Clojure data types. Source code is available from
10 http://gitweb.adaptive.cs.unm.edu/neural-net.git.
12 - a [[http://clojure.org/data_structures#Data Structures-Maps (IPersistentMap)][Map]] specifies a single neuron e.g.
14 ;; a simple neuron which sums two unit inputs
16 :accum (comp (partial reduce +) (partial map *))
20 #+begin_src latex :file data/map.pdf :border 1em :packages '(("" "tikz")) :exports none
21 \usetikzlibrary{arrows}
22 \tikzstyle{neuron} = [circle, draw, text centered, font=\footnotesize]
23 \begin{tikzpicture}[->,>=stealth', shorten >=1pt, auto]
24 \node [neuron] (neuron) at (0,0) {$\Sigma$};
25 \node (input1) at (-2,1) {$x_{1}=1$};
26 \node (input2) at (-2,-1) {$x_{2}=1$};
27 \node (weight) at (0,1) {$\overrightarrow{w}=[1,1]$};
28 \node (output) at (3,0) {$y=2$};
30 \draw (input1) -- (neuron);
31 \draw (input2) -- (neuron);
32 \draw (neuron) -- node[above] {$\Phi=identity$} (output);
37 - [[http://clojure.org/data_structures#Data Structures-Vectors (IPersistentVector)][Vectors]] are used to represent an /ordered/ series of neural elements
38 to be executed in /serial/
40 ;; two neurons in serial
41 (let [n {:phi identity
42 :accum (comp (partial reduce +) (partial map *))}
43 net [(assoc n :weights [1 1]) (assoc n :weights [2])]]
44 (run net [1 1])) ; => 4
46 #+begin_src latex :file data/vector.pdf :border 1em :packages '(("" "tikz")) :exports none
47 \usetikzlibrary{arrows}
48 \tikzstyle{neuron} = [circle, draw, text centered, font=\footnotesize]
49 \begin{tikzpicture}[->,>=stealth', shorten >=1pt, auto]
50 \node [neuron] (neuron1) at (0,0) {$\Sigma$};
51 \node [neuron] (neuron2) at (2,0) {$\Sigma$};
52 \node (input1) at (-2,1) {$x_{1}=1$};
53 \node (input2) at (-2,-1) {$x_{2}=1$};
54 \node (weight) at (-2,0) {$\overrightarrow{w}=[1,1]$};
55 \node (output) at (4,0) {$y=4$};
57 \draw (input1) -- (neuron1);
58 \draw (input2) -- (neuron1);
59 \draw (neuron1) -- node[above] {$\overrightarrow{w}=[2]$} (neuron2);
60 \draw (neuron2) -- (output);
63 [[file:data/vector.png]]
65 - [[http://clojure.org/data_structures#Data Structures-Lists (IPersistentList)][Lists]] are used to represent a /unordered/ series of neural elements
66 to be executed in /parallel/.
68 ;; a layer of three neurons in parallel followed by a single neuron
69 (let [n {:phi identity
70 :accum (comp (partial reduce +) (partial map *))
73 (run net [1 1 1])) ; => 36
75 #+begin_src latex :file data/list.pdf :border 1em :packages '(("" "tikz")) :exports none
76 \usetikzlibrary{arrows}
77 \tikzstyle{neuron} = [circle, draw, text centered, font=\footnotesize]
78 \begin{tikzpicture}[->,>=stealth', shorten >=1pt, auto]
79 \node (input1) at (-2,1) {$x_{1}=1$};
80 \node (input2) at (-2,0) {$x_{2}=1$};
81 \node (input3) at (-2,-1) {$x_{3}=1$};
82 \node [neuron] (neuron1) at (0,1) {$\Sigma$};
83 \node [neuron] (neuron2) at (0,0) {$\Sigma$};
84 \node [neuron] (neuron3) at (0,-1) {$\Sigma$};
85 \node [neuron] (neuron4) at (2,0) {$\Sigma$};
86 \node (output) at (4,0) {$y=36$};
88 \node (weight1) at (0,1.75) {$\overrightarrow{w}=[2, 2, 2]$};
89 \node (weight1) at (2,0.75) {$\overrightarrow{w}=[2, 2, 2]$};
91 \foreach \a in {1,2,3}
93 \foreach \b in {1,2,3}
95 \draw (input\a) -- (neuron\b);
97 \draw (neuron\a) -- (neuron4);
99 \draw (neuron4) -- (output);
102 [[file:data/list.png]]
104 - Graphs -- unlike the previous data types which are built into
105 Clojure Graphs are specific to this library
107 (note: this functionality is still in development, specifically
108 training may still be buggy)
110 ;; three neurons connected using an arbitrary edge set
111 (let [n {:phi identity
112 :accum (comp (partial reduce +) (partial map *))
114 g (Graph. {:a n :b n :c n} [[:a :b] [:a :c] [:b :c]])]
115 (run g [1 1 1])) ; [28]
117 #+begin_src latex :file data/graph.pdf :border 1em :packages '(("" "tikz")) :exports none
118 % {:c 28, :b 10, :a 4}
119 \usetikzlibrary{arrows}
120 \tikzstyle{neuron} = [circle, draw, text centered, font=\footnotesize]
121 \tikzstyle{value} = [text centered, font=\footnotesize]
122 \begin{tikzpicture}[->,>=stealth', shorten >=1pt, auto]
123 \node (input1) at (-2,1) {$x_{1}=1$};
124 \node (input2) at (-2,0) {$x_{2}=1$};
125 \node (input3) at (-2,-1) {$x_{3}=1$};
126 \node [neuron] (a) at (0,0.75) {$:a$};
127 \node [neuron] (b) at (1,-0.75) {$:b$};
128 \node [neuron] (c) at (2.5,0) {$:c$};
129 \node (output) at (4,0) {$y=28$};
131 \draw (input1) -- (a);
132 \draw (input2) -- (a);
133 \draw (input3) -- (b);
134 \draw (a) -- node[value] {$4$} (b);
135 \draw (a) -- node[value] {$4$} (c);
136 \draw (b) -- node[value] {$10$} (c);
137 \draw (c) -- (output);
140 [[file:data/graph.png]]
142 Example implementations are given for the following types of neural
144 - [[file:src/neural_net/perceptron.clj][perceptrons]]
145 - [[file:src/neural_net/back-propagation.clj][back propagation]] networks
146 - [[file:src/neural_net/radial-basis.clj][radial basis]] neurons /pending/
149 Copyright (C) 2010 Eric Schulte
151 This program is free software: you can redistribute it and/or modify
152 it under the terms of the GNU General Public License as published by
153 the Free Software Foundation, either version 3 of the License, or
154 (at your option) any later version.
156 This program is distributed in the hope that it will be useful,
157 but WITHOUT ANY WARRANTY; without even the implied warranty of
158 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
159 GNU General Public License for more details.
161 You should have received a [[file:COPYING][copy of the GNU General Public License]]
162 along with this program. If not, see <http://www.gnu.org/licenses/>.