Showing posts with label mathematics. Show all posts
Showing posts with label mathematics. Show all posts

Sunday, December 14, 2008

Does data have velocity?

While reading I am a Strange Loop[1] by Doug Hofstadter, where he is trying to come up with an appropriate metaphor to explain his notion of a single human "mind/soul" being distributed over multiple human brains (somewhat like a country is distributed over its many scattered embassies), it made me muse on the boundary between an actual distributed mind/soul and other mind/souls that are merely affected/influenced by that mind/soul.  This is of course a particular instance of the general problem of determining the boundary of a diffuse object.  The boundary of a solid asteroid is easy to determine whereas the borderline between one planetary ring and an adjacent ring is harder.  Any individual "rock" residing in the region where two rings overlap could be a part of either ring.

A data example of this problem is the one where lots of individual names/addresses need to be clustered into identities even though there is variation in the various names/addresses.  There are cases where it is ambiguous which identity "owns" a particular name/address when the fuzzy blob of one identity cluster overlaps the fuzzy blob of another identity. How to tell which one it belongs with? Why do we even think that there are two overlapping blobs instead of just one oddly shaped blob?


AHA - Look at velocity!

The problem of determining which points belong to which overlapping fuzzy regions is hard when looking at a static picture, however it is easy when there is movement.  When looking at which stars belong to which of two colliding galaxies, we look at the velocity of the star to see which galaxy it is moving with.

So, can this be applied to data?  Is there some "velocity" that can be determined for each data point such that it can be associated with the "proper" data cluster?  Is there a velocity associated with a name/address instance?

[1] "I am a Strange Loop",2007, Hofstadter, Basic Books

Sunday, September 9, 2007

Quantum Math for Fuzzy Ontologies

In my earlier post "Existential Programming as Quantum States", I mused that objects that were simultaneously carrying properties from multiple ontologies (i.e. multiple class hierarchies or data models), were like Quantum States in Quantum Physics.  This led me later to wonder what math had been developed to work with quantum states...i.e. is there some sort of quantum algebra that might be applicable to Existential Programming? It is needed because, in Existential Programming, a property of an object might carry multiple conflicting values simultaneously, each with varying degrees of certainty or confidence or error margins.

I found Wikipedia page on Quantum-indeterminacy which looks applicable.
Quantum indeterminacy can be quantitatively characterized by a probability distribution on the set of outcomes of measurements of an observable. The distribution is uniquely determined by the system state, and moreover quantum mechanics provides a recipe for calculating this probability distribution.
Indeterminacy in measurement was not an innovation of quantum mechanics, since it had been established early on by experimentalists that errors in measurement may lead to indeterminate outcomes. However, by the later half of the eighteenth century, measurement errors were well understood and it was known that they could either be reduced by better equipment or accounted for by statistical error models. In quantum mechanics, however, indeterminacy is of a much more fundamental nature, having nothing to do with errors or disturbance.
AHA! It dawns on me that going beyond the mere fuzzy logic idea of values having a probability or certainty factor, Existential Programming could have a fuzziness value for the property as a whole...as in "it is not certain that this property even applies to this object"...and even further it could mean "it is not certain that this property even applies to the entire Class".  A FUZZY ONTOLOGY: method of associating attributes/relationships with entities where each entity is not conclusively known. The value of a property may be certain (i.e. not vague or probabilistic), but whether that property belongs to this object is fuzzy.

Why would you want that ability?  How about data mining web pages where several people's names and a single birth-date (or phone number, address, etc) are found.  Even though it isn't known which person's name is associated with the birthday, one could associate the birth-date with each person with some fractional probability.  With enough out of focus wisps of data like this, from many web pages, the confidence factor of the right birthdate with the right person would rise to the top of the list of all possible dates (analogous to the way that very long range telescopes must accumulate lots of individual, seemingly random, photons to build up a picture of the stars/galaxies being imaged).  The fractional probability assigned could be calculated with heuristics like "lexical-distance-between-age-and-name is proportional to the probability assigned". This could make the "value" of a scalar property (like birth-date), in reality, the summarization of a complete histogram of values-by-source-web-pages.



Sunday, January 21, 2007

Imaginary Numbers paradigm for Existential Programming

It occurs to me that one of the things that Existential Programming hopes to enable is the ability to continue working with data that is vague, fuzzy, semi-inconsistent instead of screeching to a halt as would be the case with a strongly-typed implementation of a single ontology.

An analog to this is the invention (discovery?) of Imaginary Numbers in mathematics. The imaginary number "i" is defined to be the square root of -1. Now the mildly mathematical reader will note that you can't have a square root of a negative number because any time you square a number it is always positive. So, when early mathematicians came to a point in their formulas where a square root of a negative number was required, they were stuck. By creating a way to talk about and manipulate numbers that "can't exist" (i.e. imaginary numbers), formulas could be worked through such that "real" answers could eventually emerge.

By developing techniques to work with data that is not consistent with a single ontology (i.e. existential programming), programs can get past the "thats not legal data" stage and work its way to answers that ultimately do result in "legal data".

Monday, July 17, 2006

Existential Programming as Quantum States

In reading about Quantum States in Wikipedia...

"In quantum physics, a quantum state is a mathematical object that fully describes a Quantum system. One typically imagines some experimental apparatus and procedure which "prepares" this quantum state; the mathematical object then reflects the setup of the apparatus. Quantum states can be statistically mixed, corresponding to an experiment involving a random change of the parameters. States obtained in this way are called mixed states, as opposed to pure states, which cannot be described as a mixture of others. When performing a certain measurement on a quantum state, the result generally described by a probability distribution, and the form that this distribution takes is completely determined by the quantum state and the observable describing the measurement. However, unlike in classical mechanics, the result of a measurement on even a pure quantum state is only determined probabilistically. This reflects a core difference between classical and quantum physics.

Mathematically, a pure quantum state is typically represented by a vector in a Hilbert space. In physics, bra-ket notation is often used to denote such vectors. Linear combinations (superpositions) of vectors can describe interference phenomena. Mixed quantum states are described by density matrices."

...I was struck by the analogy with Existential Programming which proposes that objects hold multiple values for various properties (and in fact multiple sets of properties, hence, multiple ontologies) simultaneously.

Unlike Quantum States however, reading one set of values doesn't make the other sets vanish! ;-)

Wednesday, June 7, 2006

Ontology Merging Strategy

Summarizing the posts from yesterday, there is a general problem of "things" in one ontology/data model not mapping (in a definite way) to "things" in another model. How to support (or even automate) mapping from one model to another? I.E. how to facilitate "transformation" from one "basis" to another?

A strategy at the heart of an existential programming language could be to reduce entities to their most atomic level: semantic relations between an entity and a single attribute. Use "identity" algorithms to reconstitute these atoms into "things".

A new language that did this and integrated multiple sources of data (OO data, E/R relational data, semantic networks, web-search-results) could create a single seamless framework and data continuum.
[Ed. Note: as found 10/29/07, others have had similar ideas.]