en>Prilby: added link to Manhattan Distance (taxicab geometry) page

2015-01-11T16:02:22Z

added link to Manhattan Distance (taxicab geometry) page

128.40.255.154: /* Rescaling */

2013-11-25T13:55:52Z

Rescaling

← Older revision		Revision as of 15:55, 25 November 2013
Line 1:		Line 1:
	Hі, ~~evеrybody! Ϻy name iѕ Katja~~. <br>~~It iѕ a lіttle abοut myself~~: ~~I [http~~://~~www~~.~~free~~-~~live~~-~~sex~~-~~chat~~.~~com free live sex chat~~] in ~~Grеat Britain~~, ~~my city~~ of ~~Axford~~. <br>Іt's called ~~often Northern or cultural capital οf~~ . I'~~ve married~~ 1 ~~yearѕ ago~~.<br>~~I hɑve~~ two ~~children~~ - ~~а son~~ (~~Rodolfo~~) and the ~~daughter~~ (~~Kristeen~~). ~~Ԝe all like Driving~~.		An '''attractor network''' is a type of recurrent [[dynamical system\|dynamical]] [[network science\|network]], that evolves toward a stable pattern over time. Nodes in the attractor network converge toward a pattern that may either be fixed-point (a single state), cyclic (with regularly recurring states), [[chaos theory\|chaotic]] (locally but not globally unstable) or random ([[stochastic]]).<ref name="amit">{{cite book \| author=Amit, D. J. \| title=Modeling brain function: The world of attractor neural networks \| publisher=New York, NY: Cambridge University Press \| year=1989}}</ref> Attractor networks have largely been used in [[computational neuroscience]] to model neuronal processes such as associative memory<ref name="poucet">{{cite journal \| author=Poucet, B. and Save, E.\| title= Attractors in Memory \| journal=Science \| year=2005 \| volume=308 \| pages=799–800\| doi= 10.1126/science.1112555}}</ref> and motor behavior, as well as in [[bio-inspired computing\|biologically inspired]] methods of machine learning.
			An attractor network contains a set of ''n'' nodes, which can be represented as vectors in a ''d''-dimensional space where ''n''>''d''. Over time, the network state tends toward one of a set of predefined states on a ''d''-manifold; these are the [[attractor]]s.

			==Overview==
			{{See also\|Attractor}}
			In attractor networks, an ''attractor'' (or ''attracting set'') is a closed subset of states ''A'' toward which the system of nodes evolves. A stationary attractor is a state or sets of states where the global dynamics of the network stabilize. Cyclic attractors evolve the network toward a set of states in a [[limit cycle]], which is repeatedly traversed. Chaotic attractors are non-repeating bounded attractors that are continuously traversed.

			The network state space is the set of all possible node states. The attractor space is the set of nodes on the attractor.
			Attractor networks are initialized based on the input pattern. The dimensionality of the input pattern may differ from the dimensionality of the network nodes. The ''trajectory'' of the network consists of the set of states along the evolution path as the network converges toward the attractor state. The ''basin of attraction'' is the set of states that results in movement towards a certain attractor.<ref name="amit"/>

			==Types==
			Various types of attractors may be used to model different types of network dynamics. While fixed-point attractor networks are the most common (originating from [[Hopfield networks]]<ref name="hopfield">*{{cite journal \| author=Hopfield, J. J.\| title= Neural networks and physical systems with emergent collective computational abilities \| journal=Proceedings of the National Academy of Sciences \| year=1982 \| volume=79 \| pages=2554–2558 \| doi=10.1073/pnas.79.8.2554 \| pmid=6953413 \| pmc=346238}}</ref>), other types of networks are also examined.

			===Fixed point attractors===
			The fixed point attractor naturally follows from the [[Hopfield network]]. Conventionally, fixed points in this model represent encoded memories. These models have been used to explain associative memory, classification, and pattern completion. Hopfield nets contain an underlying [[Lyapunov function\|energy function]]<ref name="shopfield">*{{Scholarpedia\|title=Hopfield network\|curator=John Hopfield\|urlname= Hopfield_network}}</ref> that allow the network to asymptotically approach a stationary state. One class of point attractor network is initialized with an input, after which the input is removed and the network moves toward a stable state. Another class of attractor network features predefined weights that are probed by different types of input. If this stable state is different during and after the input, it serves as a model of associative memory. However, if the states during and after input do not differ, the network can be used for pattern completion.

			===Other stationary attractors===
			Line attractors and plane attractors are used in the study of oculomotor control. These line attractors, or ''neural integrators'', describe eye position in response to stimuli. Ring attractors have been used to model rodent head direction.

			===Cyclic attractors===
			Cyclic attractors are instrumental in modelling [[central pattern generator]]s, neurons that govern oscillatory activity in animals such as chewing, walking, and breathing.

			===Chaotic attractors===
			Chaotic attractors (also called ''strange attractors'') have been hypothesized to reflect patterns in odor recognition. While chaotic attractors have the benefit of more quickly converging upon limit cycles, there is yet no experimental evidence to support this theory.<ref name="attractor">*{{Scholarpedia\|title=Attractor network\|curator=Chris Eliasmith\|urlname=Attractor_network}}</ref>

			==Implementations==
			Attractor networks have mainly been implemented as memory models using fixed-point attractors. However, they have been largely impractical for computational purposes because of difficulties in designing the attractor landscape and network wiring, resulting in spurious attractors and poorly conditioned basins of attraction. Furthermore, training on attractor networks is generally computationally expensive, compared to other methods such as [[k-nearest neighbor]] classifiers.<ref name="zemel">*{{cite journal \| author=Zemel, R. and Mozer, M.\| title= Localist attractor networks \| journal=Neural Computation \| year=2001 \| volume=13 \| pages=1045–1064}}</ref>

			===Hopfield Networks===
			{{See also\|Hopfield network}}
			Hopfield attractor networks are an early implementation of attractor networks with associative memory. These recurrent networks are initialized by the input, and tend toward a fixed-point attractor. The update function in discrete time is <math>x(t+1)=f(Wx(t))</math>, where <math>x</math> is a vector of nodes in the network and <math>W</math> is a symmetric matrix describing their connectivity. The continuous time update is <math>\frac{dx}{dt}=-\lambda x+f(Wx)</math>.

			''[[Bidirectional associative memory\|Bidirectional networks]]'' are similar to Hopfield networks, with the special case that the matrix <math>W</math> is a [[block matrix]].<ref name="shopfield"/>

			===Localist Attractor Networks===
			Zemel and Mozer (2001)<ref name="zemel"/> proposed a method to reduce the number of spurious attractors that arise from the encoding of multiple attractors by each connection in the network. Localist attractor networks encode knowledge locally by implementing an [[expectation-maximization]] algorithm on a [[mixture model\|mixture-of-gaussians]] representing the attractors, to minimize the free energy in the network and converge only the most relevant attractor. This results in the following update equations:
			#Determine the activity of attractors: <math>q_i(t)=\frac{\pi_i(y(t),w_i,\sigma(t))}{\sum_j\pi_jg(y(t),w_j,\sigma(t))}</math>
			#Determine the next state of the network: <math>y(t+1)=\alpha(t)\xi+(1-\alpha(t))\sum_iq_i(t)w_i\,\!</math>
			#Determine the attractor width through network: <math>\sigma^2_y(t)=\frac{1}{n}\sum_iq_i(t)\|y(t)-w_i\|^2</math>
			(<math>\pi_i</math> denotes basin strength, <math>w_i</math> denotes the center of the basin. <math>\xi</math> denotes input to the net.)

			The network is then re-observed, and the above steps repeat until convergence. The model also reflects two biologically relevant concepts. The change in <math>\alpha</math> models stimulus ''[[Priming (psychology)\|priming]]'' by allowing quicker convergence toward a recently visited attractor. Furthermore, the summed activity of attractors allows a ''gang effect'' that causes two nearby attractors to mutually reinforce the other's basin.

			===Reconsolidation Attractor Networks===
			Siegelmann (2008)<ref name="siegelmann">*{{cite journal \| author=Siegelmann, H. T.\| title= Analog-symbolic memory that tracks via reconsolidation \| journal=Physica D\| year=2008 \| volume=237 \| pages=1207–1214}}</ref> generalized the localist attractor network model to include the tuning of attractors themselves. This algorithm uses the EM method above, with the following modifications: (1) early termination of the algorithm when the attractor's activity is most distributed, or when high entropy suggests a need for additional memories, and (2) the ability to update the attractors themselves: <math>w_i(t+1)=vq_i(t)\cdot y(t)+[1-vq_i(t)]\cdot w_i(t)\,\!</math>, where <math>v</math> is the step size parameter of the change of <math>w_i</math>. This model reflects [[memory reconsolidation]] in animals, and shows some of the same dynamics as those found in memory experiments.

			Further developments in attractor networks, such as [[kernel trick\|kernel]] based attractor networks,<ref>*{{cite journal \| author=Nowicki, D., Siegelmann, H.T.\| title= Flexible Kernel Memory \| journal=PLoS ONE \| year=2010 \| volume=5 \| issue=6\|pages=e10955}}</ref> have improved the computational feasibility of attractor networks as a learning algorithm, while maintaining the high-level flexibility to perform pattern completion on complex compositional structures.

			==References==
			{{Reflist}}



			[[Category:Networks]]

en>ChrisGualtieri: /* References */Typo fixing, typos fixed: , → , (2) using AWB

2012-08-11T01:40:02Z

References: Typo fixing, typos fixed: , → , (2) using AWB

New page

Hі, evеrybody! Ϻy name iѕ Katja. <br>It iѕ a lіttle abοut myself: I [http://www.free-live-sex-chat.com free live sex chat] in Grеat Britain, my city of Axford. <br>Іt's called often Northern or cultural capital οf . I've married 1 yearѕ ago.<br>I hɑve two children - а son (Rodolfo) and the daughter (Kristeen). Ԝe all like Driving.

Feature scaling - Revision history

en>Prilby: added link to Manhattan Distance (taxicab geometry) page

128.40.255.154: /* Rescaling */

en>ChrisGualtieri: /* References */Typo fixing, typos fixed: , → , (2) using AWB