Comments on: Atoms of English http://kybernetikos.com/2007/12/03/atoms-of-english/ Tue, 07 Feb 2012 23:28:52 +0000 http://wordpress.org/?v=2.0.2 by: kyb http://kybernetikos.com/2007/12/03/atoms-of-english/#comment-6114 Sun, 09 Dec 2007 20:03:30 +0000 http://kybernetikos.com/2007/12/03/atoms-of-english/#comment-6114 Update: I noticed that some of my parsing was off - most of the words were fine, but some were a bit messed up. I'm not going to bother rerunning it just now, since it takes ages. Anyway, I'm more interested in visualising the graph... John: Do you want the list for programming with, or for looking at? And how is sorting it according to java hashcode not useful? :-) Actually, you've made me want to publish a dictionary where the words are on the page of their hashcode according to a simple human computable method. I ran a couple of tests with features humans could easily spot (number of descenders, number of vowels, etc) and then modding it by the intended number of pages (500 seems reasonable), but I'm still not down to a reasonable maximum number of words per page. If anyone can help me, we can share the enormous profits from the publishing (lulu). Fantastic, a dictionary you can only use if you already know how to spell the word and can do mental arithmetic. Could make it even more evil if you included aspects of the meaning into the hash code, then you could only use it if you know what the word means and how to spell it. You know it'd be a hit with elitist geeks. Update: I noticed that some of my parsing was off - most of the words were fine, but some were a bit messed up. I’m not going to bother rerunning it just now, since it takes ages. Anyway, I’m more interested in visualising the graph…

John: Do you want the list for programming with, or for looking at? And how is sorting it according to java hashcode not useful? :-) Actually, you’ve made me want to publish a dictionary where the words are on the page of their hashcode according to a simple human computable method. I ran a couple of tests with features humans could easily spot (number of descenders, number of vowels, etc) and then modding it by the intended number of pages (500 seems reasonable), but I’m still not down to a reasonable maximum number of words per page. If anyone can help me, we can share the enormous profits from the publishing (lulu).

Fantastic, a dictionary you can only use if you already know how to spell the word and can do mental arithmetic. Could make it even more evil if you included aspects of the meaning into the hash code, then you could only use it if you know what the word means and how to spell it. You know it’d be a hit with elitist geeks.

]]> by: John http://kybernetikos.com/2007/12/03/atoms-of-english/#comment-5985 Wed, 05 Dec 2007 14:03:44 +0000 http://kybernetikos.com/2007/12/03/atoms-of-english/#comment-5985 Would it be possible to sort the list alphabetically? Would it be possible to sort the list alphabetically?

]]> by: kyb http://kybernetikos.com/2007/12/03/atoms-of-english/#comment-5942 Tue, 04 Dec 2007 11:13:28 +0000 http://kybernetikos.com/2007/12/03/atoms-of-english/#comment-5942 I would have liked a dictionary that gave me word trunks, but ideally in English. You can have the java source if fancy trying it on German. However, I'm not totally sure that all derived versions of words are necessarily redundant. You can never predict 100% the way a derived word will be used just from the derivation, so there is some level of distinctiveness in each derived word, and that distinctiveness may include an atomic concept. However I'm not claiming this technique is even reasonable, just the best I could come up with in a few hours the other night... I'm planning to do a directed graph visualisation of all the words needed to define other words at some point, and I'll post it here when I do. Have to finish detecting edges first though. I would have liked a dictionary that gave me word trunks, but ideally in English. You can have the java source if fancy trying it on German. However, I’m not totally sure that all derived versions of words are necessarily redundant. You can never predict 100% the way a derived word will be used just from the derivation, so there is some level of distinctiveness in each derived word, and that distinctiveness may include an atomic concept.

However I’m not claiming this technique is even reasonable, just the best I could come up with in a few hours the other night…

I’m planning to do a directed graph visualisation of all the words needed to define other words at some point, and I’ll post it here when I do. Have to finish detecting edges first though.

]]> by: woodly http://kybernetikos.com/2007/12/03/atoms-of-english/#comment-5930 Tue, 04 Dec 2007 07:25:40 +0000 http://kybernetikos.com/2007/12/03/atoms-of-english/#comment-5930 what school of thought are you referencing exactly? "blow" and "blown" are redundant for example, you should work on word trunks like canoo.net :-) what school of thought are you referencing exactly?
“blow” and “blown” are redundant for example, you should work on word trunks like canoo.net :-)

]]>