Hopbot log for 2008-02-10 - Helma IRC channel: #helma on irc.freenode.net
2008-02-10:
[4:13] <jkridne1> zumbrunn: that's a relief. I guess you have made something relatively fool proof. :)[4:14] <jkridne1> I'm confused where the collection/children information goes. it doesn't seem to show up in the database.
[4:15] <jkridner> is it all going to the XML object stores?
[4:15] <jkridner> I'm concerned that my database lookups won't be at all efficient.
[8:18] <zumbrunn> jkridner, yt?
[8:18] <zumbrunn> for the Page objects you have defined type.properties that would map page objects to the relational db
[8:19] <zumbrunn> so, these should show up in that db and should not be written to the embedded xml db
[8:20] <zumbrunn> otherwise you have something configured wrong somewhere
[8:22] <zumbrunn> you can check whether the data is being written to the xml db by taking a look inside the ./db/... subdirectory with the name of your application
[8:25] <zumbrunn> if your code just attaches data to a plain js object property then it is also possible that the data is simply cached in memory, but would disappear when you flush the cache or restart the app.
[8:26] <zumbrunn> which collection/children are you talking about?
[8:27] <zumbrunn> the ones defined in the Page prototype?
[15:30] <jkridner> zumbrunn: we seem to have very different schedules right now. :)
[15:30] <zumbrunn> :-)
[15:30] <jkridner> oh, hi!
[15:30] <zumbrunn> with some overlap, as you can see ;-)
[15:31] <jkridner> Indeed.
[15:31] <jkridner> I have some .xml files in the db directory.
[15:32] <zumbrunn> and what kinds of objects does that concern?
[15:32] <zumbrunn> the children collection of the Page prototype?
[15:32] <jkridner> I can't quite figure out the order of what is in there, since the are various types of objects all numbered 0..9.xml
[15:33] <jkridner> er, 0..7.xml
[15:33] <zumbrunn> 0.xml is the root object
[15:33] <jkridner> As I've been editing my program, I've been changing what the root's children are.
[15:33] <jkridner> oh, actually...
[15:33] <jkridner> it looks like 1.xml is just a link to the others...
[15:34] <jkridner> and 2..7.xml are some old children, not what the app is using today.
[15:34] <zumbrunn> ok, they are probably just still in there from your earlier tests
[15:35] <zumbrunn> but your get no data added inside your relational db?
[15:35] <jkridner> so, it must all be in the database, but I don't see that I have the right indexes or any references from root.
[15:35] <jkridner> I do have data in my relational db, but I don't see the references to say which objects are the children of root.
[15:36] <jkridner> the way my app is meant to work is that the objects shouldn't be deleted when a replacement is made...
[15:36] <jkridner> instead, the new object is added as a child to root and the old object is made a child of the new object.
[15:36] <zumbrunn> ok, that should be possible
[15:37] <jkridner> the SQL dump is actually visible at the gitweb link I provided earlier.
[15:37] <jkridner> I don't see any indexing or primary key.
[15:37] <zumbrunn> yeah, but I suck at that since I always just use the built-in xml db myself
[15:38] <jkridner> I'm a little more used to MySQL than PostgreSQL.
[15:38] <zumbrunn> not relational backends
[15:38] <jkridner> It seems easier, but I thought the performance would be better if I used the relational backend.
[15:38] <zumbrunn> can you connect to the db using the sqlshell?
[15:38] <jkridner> I can with psql. It is postgres.
[15:38] <jkridner> oh..
[15:38] <jkridner> you mean the Helma tool.
[15:39] <zumbrunn> http://helma.zumbrunn.com/tools/about_sqlshell
[15:42] <jkridner> k, I've started sqlshell
[15:43] <zumbrunn> if you use the same db.properties, you should be able to browse the tables in your db using the sqlshell
[15:43] <zumbrunn> (you should be able to pick your db as the datasource)
[15:44] <jkridner> yes, that is working.
[15:45] <zumbrunn> but if you add a new Page object, it is not persisted in the db?
[15:45] <jkridner> it is. no problems there.
[15:46] <zumbrunn> so, what isn't working?
[15:46] <jkridner> I'm just confused about where the children pointers are...
[15:47] <jkridner> because all of the old Page objects stay in the same db table.
[15:47] <jkridner> somehow, root.get() seems to find the right one, but I don't know how looking at the database.
[15:47] <jkridner> I'm using the 'uri' field as the index
[15:48] <zumbrunn> is root itself persisted in the relational db?
[15:48] <jkridner> but the old Page objects have the same 'uri' value.
[15:48] <zumbrunn> I don't remember what you had in the Root type.properties
[15:48] <jkridner> root is a different type of object.. a HopObject or a Root object, right?
[15:49] <zumbrunn> by default, it is of prototype Root and inherits from HopObject, yes
[15:49] <jkridner> I just have the _children = collection(Page) and _children.access=uri
[15:49] <zumbrunn> unless you set an _extends=Foo property in Root's type.properties
[15:50] <zumbrunn> then I think it would just store the ids in the 0.xml file
[15:50] <zumbrunn> (list the children there)
[15:51] <jkridner> k, that is because I haven't declared a db for the Root object type? I must be looking at the wrong db directory then.
[15:51] <zumbrunn> yes
[15:51] <zumbrunn> (assuming you didn't map the HopObject prototype to a db table)
[15:51] <jkridner> so, the other concern I have is the Page objects in the database don't seem to be index.
[15:52] <jkridner> I'm pretty sure I didn't, but I'll check.
[15:53] <jkridner> I don't even have a HopObject in my application's directory.
[15:53] <zumbrunn> ok
[15:54] <jkridner> k, so I'll look around for where the right .xml files are being created. I must have moved the running directory of Helma.
[15:54] <jkridner> right now I'm having some sort of strange network issue where my connection is very slow.
[15:58] <jkridner> is there any way to easily know where the 'db' directory is that is currently being used?
[15:59] <jkridner> I have a few Helma installations as I was playing with Trimpath Junction. :)
[15:59] <zumbrunn> it should definitely be the dir with your app name inside ./db/...
[16:00] <zumbrunn> unless you are setting a custom location in app.properties or server.properties
[16:00] <zumbrunn> (which, actually, Trimpatch Junction does, I believe)
[16:01] <zumbrunn> Trimpath
[16:02] <zumbrunn> that would be a dbHome property
[16:02] <zumbrunn> http://helma.org/docs/guide/properties/listall/
[16:03] <zumbrunn> and hopHome defines what ./ would mean
[16:09] <jkridner> I can't find any redefinition statements, but the objects are all old.
[16:10] <zumbrunn> if you restart the app, does helma still find the Page objects that are children from root?
[16:11] <zumbrunn> you could stop helma, move the db/myapp dir somewhere safe and start helma again
[16:11] <zumbrunn> then it has to create a new db dir and new 0.xml file and you'll see if it has any effect at all
[16:11] <jkridner> I'm scared to find out. Best to try while the database is still small and easy to rebuild. :)
[16:12] <zumbrunn> back up the db as well, then :-)
[16:12] <jkridner> indeed. :)
[16:12] <jkridner> looks like I'll have to rebuild the links anyway, because I suspect that I'll lose them.
[16:13] <zumbrunn> I would think that you do
[16:13] <zumbrunn> but you'll find out
[16:13] <zumbrunn> if they are gone, stop helma again...
[16:14] <jkridner> k
[16:14] <zumbrunn> just copy the old 0.xml file into your new db/myapp/ dir...
[16:14] <zumbrunn> and start helma gain and see whether the children are there again
[16:15] <jkridner> they are still there.
[16:16] <jkridner> still, the 'db' directory that I think is right shows the wrong type of objects.
[16:16] <jkridner> your system is quite resilient, I just can't understand it. :)
[16:16] <zumbrunn> :-)
[16:16] <zumbrunn> then maybe you defined a collection that simple adds all records in your Page table that point back to Root as their parent
[16:17] <jkridner> what does that buy me?
[16:18] <zumbrunn> if you defined the children of Root to be a collection of Page objects...
[16:19] <jkridner> but that is what it is defined as.
[16:19] <zumbrunn> then helma goes and checks your db table for page objects that point back at Root as their parent
[16:20] <zumbrunn> does that make any sense?
[16:20] <jkridner> localhost Root # cat type.properties
[16:20] <jkridner> _children = collection(Page)
[16:20] <jkridner> _children.accessname = uri
[16:20] <jkridner> _children.order = uri asc
[16:20] <jkridner> sure, but which record in the table would be used to do that?
[16:20] <jkridner> are you suggesting I create one?
[16:23] <zumbrunn> that's what the _parent mapping would be for, I believe
[16:24] <jkridner> so, that would let me find the reference, but Helma is apparently generating one somewhere. I don't want to wear out my welcome. You've been very helpful. I really appreciate it.
[16:25] <jkridner> I have learned a lot with this.
[16:25] <zumbrunn> you don't wont all Page objects to automatically be children of Root, or do you?
[16:26] <jkridner> no, I don't.
[16:26] <zumbrunn> I suspect that currently you get automatically all of them
[16:26] <zumbrunn> (but I'm not sure)
[16:26] <jkridner> oh.
[16:26] <jkridner> so, I most likely need a '_parent' field.
[16:26] <zumbrunn> that's my guess, yes
[16:29] <jkridner> so, I put a field in the 'db' for that, but I'm not currently using it. I called it 'prev', as a backreference. I guess I should add _parent = prev?
[16:29] <jkridner> it is the same type (integer) as the _id.
[16:29] <jkridner> it otherwise isn't in my current type.properties.
[16:30] <zumbrunn> don't know if the type matters
[16:32] <zumbrunn> (I don't mind the chatting by the way)
[16:33] <zumbrunn> I'm just not sure how helpful I am or whether i just confuse you more
[16:33] <jkridner> That's good. I expect to be back many times. :)
[16:33] <jkridner> you haven't confused me at all. you have been very helpful.
[16:33] <zumbrunn> but it's one way for me to learn about this stuff without actually doing it myself
[16:33] <zumbrunn> ;-)
[16:34] <jkridner> I wish that there weren't .properties files...
[16:34] <jkridner> I much prefer to do everything in one language.
[16:34] <jkridner> which is part of why I'm using Helma to begin with.
[16:34] <jkridner> JS on server and client.
[16:41] <jkridner> k, once I add this, I still won't know how it is working now. I think those references are being stored somewhere. Otherwise, what might be happening is simply that the first object found is always the most recent.
[16:41] <jkridner> if I add _parent = prev, how does that work when pointing to Root?
[16:41] <jkridner> since Root doesn't have the same type of id?
[16:46] <zumbrunn> sorry, I was afk for a while
[16:46] <jkridner> here's another strange thing to me, Helma doesn't seem to have a problem with me creating children that aren't of the type specified by _children. For instance, my code still seems to add User objects to root when I do app.registerUser.
[16:47] <zumbrunn> it does? it should add them to a special user object
[16:48] <zumbrunn> (separate from root)
[16:49] <jkridner> well, maybe I'm just confused then. Those are the objects I'm seeing.
[16:49] <jkridner> I guess I don't know that they are under Root.
[16:51] <jkridner> I guess you are right. the parent object is 'name="users"'.
[16:51] <jkridner> I guess this doesn't conflict with my object called Users?
[16:51] <jkridner> I guess that it does actually.
[16:52] <zumbrunn> changing things so type properties could be configured in code has been suggested, but it's not yet clear when this will happen
[16:52] <zumbrunn> I don't believe it conflicts with Users since the built-in one is called User
[16:53] <jkridner> I'm going to try 'shell' now to look at this.
[16:53] <zumbrunn> oh, the *name* of it might conflict
[16:53] <zumbrunn> I was thinking about the prototype "User"
[16:53] <jkridner> ah, this tells the story.
[16:54] <jkridner> sorry, I'm back to my original problem...
[16:54] <jkridner> I think 'shell' tells the whole story...
[16:54] <jkridner> they are all indeed children of Root...
[16:54] <jkridner> I'm just getting lucky to pull the latest one out.
[16:56] <zumbrunn> strange, did you define a User prototype with any type.properties?
[16:56] <jkridner> I don't see the 'user' objects under shell though.
[16:56] <zumbrunn> by default that's not what I would expect
[16:56] <zumbrunn> ok, I'm confused now
[16:56] <jkridner> I have a User object, but no type.properties.
[16:58] <zumbrunn> usually, without type.properties that map these prototypes to a relational db, you would get 0.xml as the root object and 1.xml as the index of users
[17:00] <jkridner> that seems to be what is happening... I was thinking that was old information, but I can see now that it is app.registerUser that is doing this.
[17:01] <jkridner> is there a way to look at those objects with 'shell'?
[17:01] <jkridner> they don't seem to show up as children of Root.
[17:02] <zumbrunn> which objects? the individual user objects?
[17:02] <jkridner> yes.
[17:02] <zumbrunn> http://helma.zumbrunn.com/reference/app.html#app.getUser
[17:03] <zumbrunn> http://helma.zumbrunn.com/reference/HopObject.html#HopObject.getById
[17:04] <jkridner> it returns in the shell, but I don't see a way to explore it like I'm exploring [object Root].
[17:04] <zumbrunn> yes, you'ld have to script that right now, i believe
[17:05] <zumbrunn> we could make the shell smarter so that this would be possible
[17:05] <zumbrunn> kind of combine it with the functionality of the HopRepl, which would allow you to do that now, I think
[17:05] <jkridner> it would be handy, since the shell is almost a complete view of the object store. It is much easier for me to comprehend than the sqlshell.
[17:06] <zumbrunn> http://dev.helma.org/wiki/HopRepl/
[17:07] <zumbrunn> if you add a mountpoint for the User prototype, you would then have a URL where you could use the shell on user objects as well
[17:08] <zumbrunn> http://helma.org/docs/guide/mapping/mountpoint/
[17:08] <zumbrunn> users = mountpoint(User)
[17:08] <jkridner> you just flipped on a light-switch there. I'd missed that meaning of mountpoint.
[17:09] <jkridner> I'd been struggling to figure out how to mount everything off of root and children.
[17:10] <jkridner> btw, I like the shell in the browser. I think I'd just want to add some references to objects that aren't mounted.
[17:10] <jkridner> i added my _parent = prev line, but it seems I must restart helma for that change to take effect.
[17:12] <jkridner> where does the user = mountpoint(User) go?
[17:13] <zumbrunn> in the type.properties for Root, I was thinking
[17:17] <jkridner> hurray! The _parent setting is now giving me what I expect. I'm now really creating children w/in the database.
[17:20] <zumbrunn> midnightmonster to the rescue!
[17:21] <jkridner> hmmm... not quite. I'm still a tiny bit confused.
[17:22] <zumbrunn> if the mountpoint thing isn't working, it's probably because midnightmonster was right when he recently claimed the docs are wrong
[17:22] <zumbrunn> http://helma.zumbrunn.net/hopbot/2008-01-28
[17:22] <zumbrunn> (after 14:00)
[17:23] <jkridner> the behavior is right... http://beagleboard.org/helma_test/helma_test/helma_test
[17:23] <jkridner> http://beagleboard.org/helma_test/helma_test
[17:23] <jkridner> http://beagleboard.org/helma_test/helma_test
[17:24] <jkridner> http://beagleboard.org/helma_test
[17:24] <jkridner> k, for new objects it all seems to work fine. there is a bit of confusion in the database for objects that already existed without the _parent field defined.
[17:25] <midnightmonster> If I'm expected to be useful now, can someone provide a quick summary? y'all been talking a while
[17:25] <zumbrunn> I'm just giving advice on stuff I I would really need advice myself
[17:25] <jkridner> I've been trying to understand how my objects were being found...
[17:25] <jkridner> because I have a table and couldn't find the references.
[17:26] <zumbrunn> and things midnightmonster probably knows more about than I do
[17:26] <jkridner> what was actually happening was that the first object where there was a match was being pulled out and it happened to be the right one.
[17:26] <zumbrunn> so, I was just glad to see you join
[17:26] <midnightmonster> ok :-)
[17:27] <jkridner> zumbrunn advised me to add '_parent = field' and now I'm no longer mystified as to how the right object is being pulled out.
[17:27] <midnightmonster> jkridner, are you working on some sort of versioning for a wiki-like thing?
[17:27] <jkridner> yes, exactly.
[17:27] <midnightmonster> I've been pondering much the same
[17:27] <jkridner> Gobi/Gobble/etc. were too complex for me.
[17:28] <midnightmonster> I just have a bad case of NIH syndrome
[17:28] <jkridner> I wanted something simpler and I have a few new requirements, namely supporting OpenID.
[17:29] <jkridner> http://beagleboard.org/ source at http://www.beagleboard.org/gitweb/?p=beagleboard.org.git
[17:29] <midnightmonster> I've implemented arbitrary-depth wikiness, so /foo and /foo/bar, so I was thinking of putting past revisions in a PageHistory (collection-type HopObject) property of my pages. (direct children are the next level of page)
[17:31] <jkridner> when I get a page edit, I'm taking the old object and adding it to the new object, then removing it from root while adding the new object as a child of root.
[17:32] <jkridner> I'm also going to make collections for alternate languages.
[17:32] <midnightmonster> I got that. I haven't implemented that yet--just saying that when I do I'll need to attach my old versions to some other property of my pages than being direct children
[17:32] <jkridner> the original page created would hold the alternate languages as children.
[17:33] <jkridner> why would they need to be some other property, rather than just children?
[17:33] <jkridner> oh...
[17:33] <midnightmonster> 'cause my wiki can have /foo and then /foo/bar, where bar is the child of foo
[17:33] <jkridner> because you are using direct children as deeper...
[17:33] <jkridner> right.
[17:33] <jkridner> sorry, just caught that.
[17:34] <jkridner> I guess that would make sense for me too, but I decided I didn't need depth > 1 for the page URIs.
[17:34] <midnightmonster> I read a really good slide deck about how someone in python-land set up their relational schema to efficiently (storage and speed) handle revisions and languages in their cms...
[17:34] <midnightmonster> bookmark's on the other computer. I'll see if I can find it in a few
[17:35] <jkridner> part of what brought me here is that I was trying to figure out how to get the various sorts that I need.
[17:35] <jkridner> clearly I need a fast index into the pages that are children of root based on their name. I also need a fast sort by ID to get my parent references.
[17:36] <midnightmonster> those aren't sorts... just lookups
[17:36] <midnightmonster> and postgres will take care of that for you trivially with one extra index
[17:37] <midnightmonster> assuming that you're using an int with a sequence as the primary key, and that as your _id?
[17:38] <jkridner> that's pretty much what I'm doing (_id = id)
[17:38] <midnightmonster> (helma's object db is also very very fast for those things for small-ish totals of objects. so far I haven't hit the limit, but I'm not pushing it hard)
[17:38] <jkridner> but, postgres hasn't created a primary key for it.
[17:38] <midnightmonster> if you called it a primary key when you created the table it did
[17:39] <midnightmonster> how are you talking to postgres?
[17:39] <jkridner> should I dump postgres until I've solidified what my db looks like and I really need optimization?
[17:39] <jkridner> what do you mean by how I'm talking to postgres? it is on the same machine using JDBC.
[17:39] <midnightmonster> I kinda like h2 for fast helma dev, 'cause it's a fast java db that can run in-process with helma or apart if you need
[17:40] <jkridner> I didn't make it the primary key when I created the table.
[17:40] <midnightmonster> also comes with a really sweet web console
[17:40] <midnightmonster> you can make it a primary key after the fact with a sql command and also add an index on the name
[17:41] <jkridner> with either db?
[17:41] <midnightmonster> but if you like mysql (didn't you say that?) why'd you switch?
[17:41] <midnightmonster> yes, with any reasonable sql db--and some that aren't so reasonable
[17:41] <jkridner> do you find prototyping a lot easier w/o the relational backend? I'm thinking I want to strip it out until I'm ready to optimize.
[17:41] <jkridner> I'm a novice with all of them.
[17:42] <jkridner> so far, I like postgres and I don't even remember why I switched...
[17:42] <midnightmonster> I do think it's faster without the relational db.
[17:42] <jkridner> it had to do with some examples I was trying to follow.
[17:42] <jkridner> is there a quick way to copy/backup objects?
[17:42] <midnightmonster> (postgres' better. but not so much better that it would probably be worth switching for something like this if you already knew one reasonably well)
[17:43] <midnightmonster> quick way to copy/backup in what context?
[17:43] <jkridner> I've used MySQL more before, but I don't know it well enough to be tied to it.
[17:44] <jkridner> just to switch databases.
[17:44] <jkridner> I only have about a dozen records anyway.
[17:45] <midnightmonster> you should be able to go to several thousand objects without the slightest problem in Helma's object store. I don't know beyond that, 'cause I've not tried
[17:45] <jkridner> I can copy/paste from the dumps.
[17:45] <jkridner> I will have thousands before I'm done.
[17:47] <midnightmonster> my plan is to see how far the xml db will go and figure out an alternative or a way to improve the xml db if/when I need it
[17:48] <midnightmonster> there is not, however, afaict, an easy built-in way to move one's data from xml-db to a relational db. I figure if I have to do it I can hack something.
[17:52] <jkridner> yeah, I think I can move back to the XML db and cross that bridge when I reach it as well.
[17:55] <jkridner> ouch. I decided to bite the bullet and move over (to give me better visibility as I'm building this up) and I've run into a null pointer exception.
[17:55] <jkridner> the function is just root.add(x).
[17:55] <jkridner> where x = new Page().
[17:56] <midnightmonster> if you won't be missing anything to do it, I would kill the app/helma, delete the contents of the db directory, and start over
[17:56] <midnightmonster> just restarting the app might do it, though, if you didn't do that already
[18:00] <jkridner> yeah, the problem seems to be gone.
[18:01] <midnightmonster> you'll get those from inconsistencies in the xml db, which can happen if you screw something up with manually fiddling with the db. you can fix it manually, too, but it's a pain.
[18:01] <jkridner> I'll have to give some thoughts to making the history collections rather than children.
[18:02] <jkridner> I wonder how changing something like that on the fly would work. I'm afraid I may end up rebuilding the db often at this rate. :)
[18:03] <jkridner> I think I might want to create some methods to handle hat.
[18:03] <jkridner> that.
[18:15] <jkridner> I just found a nasty with the XML db that I don't get...
[18:16] <jkridner> root.removeChild(this) seems to have removed my object completely, rather than just the link.
[18:16] <jkridner> actually, I'm speaking too fast... I need to see why my object went away.
[18:16] <midnightmonster> shouldn't have done
[18:17] <midnightmonster> but if you find that's a problem, you might try adding it to the new parent before removing from the old
[18:23] <jkridner> looks like it is an error in my form creation on the page. When I was using the database and my parent links weren't right, the error didn't show. Now, my app really does understand the difference between 'create' and 'edit' actions and that is what screwed me up.
[18:23] <jkridner> I need to fix my 'action' macro.
[18:25] <jkridner> currently http://beagleboard.org/default/create shows 'action="edit"'.
[18:26] <midnightmonster> I don't use the macros at all... so I'm not as useful on that
[18:26] <jkridner> easy fix.
[18:26] <jkridner> I just changed the skin... seems easier.
[18:29] <jkridner> I'm going to go ahead with your approach of using a history collection, rather than children. seems like it'll give me more flexibility for the future.
[18:30] <zumbrunn> one downside of keeping the different steps in the history as separate objects is that it artificially inflates the number of objects in the db
[18:30] <jkridner> awesome. The more I understand this stuff, it really gets fun.
[18:31] <jkridner> there are a couple of conceptual minefields, but it really make some big things pretty darn simple.
[18:31] <zumbrunn> I'm also going to need to do something like this for jhino
[18:31] <midnightmonster> zumbrunn, since one cannot store array's directly, do you see a workaround?
[18:31] <zumbrunn> and I was planning to do it as one hopobject for the entire history
[18:32] <midnightmonster> and if the objects are rarely fetched, it shouldn't make much difference that they're hanging out in the db, no?
[18:32] <zumbrunn> or something like 100 steps in one history object
[18:32] <midnightmonster> how would you store history in one object?
[18:32] <jkridner> true, it would be nice to just store a diff in a single object or something.
[18:32] <jkridner> as a diff?
[18:32] <zumbrunn> no, probably the actual content, but in separate properties
[18:33] <jkridner> updating the references seems like it'd be really fast. I don't see why performance would degrade much... just a lot of storage.
[18:33] <zumbrunn> right
[18:34] <midnightmonster> zumbrunn, so like history.rev##TIMESTAMP## = "my last content"?
[18:34] <zumbrunn> yes, something like that
[18:34] <zumbrunn> or just by the rev number
[18:35] <midnightmonster> or this.history.rev...
[18:37] <zumbrunn> I was thinking I'll just create a js object with properties for the data I want to save, serialize it to json and store it in the history object as a property named using the appropriate rev number
[18:38] <midnightmonster> makes sense
[18:38] <zumbrunn> that way the history consumes only one (or only a few) objects, but is disconnected from the current rev of the object and doesn't need to get loaded into the cache unless it is really needed
[18:40] <midnightmonster> on the downside, if you need any item of the history you have to load into memory the whole friggin thing
[18:40] <zumbrunn> right, but that wouldn't happen very often, I was thinking
[18:41] <jkridner> well, you convinced me to put my history in a collection, rather than as children, but I think I'll stick with that and not put it all into one property for now.
[18:42] <jkridner> I do wonder if I want all revisions to show up under the same collection...
[18:42] <jkridner> right now, I have more of a single-link list.
[18:42] <midnightmonster> why not?
[18:42] <jkridner> I can't think of a good reason, but the current method maintains the order of the edits.
[18:43] <jkridner> I do have timestamps, however.
[18:43] <midnightmonster> collections are ordered
[18:44] <jkridner> If I put a symlink in my app directory to the XML database, will it screw anything up?
[18:44] <midnightmonster> no, but why?
[18:44] <jkridner> true. I guess the logic is simple enough, but wouldn't I have to walk the old collection?
[18:44] <jkridner> because I want to do that to make my version control system pick up the database.
[18:45] <jkridner> currently I'm versioning only my application code, but I want to start saving off the database as well, at least for the time being.
[18:45] <midnightmonster> helma reads the whole app dir and descendants looking for executable stuff, so I wouldn't put it there
[18:46] <jkridner> right, that's what I was concerned about, but I don't think anything there would have a .hac/.js/.skin/etc. extension.
[18:46] <zumbrunn> if by "app directory" you mean the main code repository, then helma would think your db dir is a prototype
[18:47] <midnightmonster> I put my stuff in /appname/app, /appname/db, etc
[18:47] <jkridner> k. Guess I'll need to find some way to have 'git' catch that otherwise.
[18:47] <midnightmonster> and keep /appname under version control
[18:47] <jkridner> I made the mistake of making my repository the root of the app.
[18:47] <midnightmonster> just move it all to a subfolder
[18:48] <jkridner> I can't say I'm fond of all the autodirectory scanning stuff.
[18:48] <midnightmonster> alternative is you'd have to tell helma where each thing was
[18:48] <midnightmonster> pita
[18:48] <jkridner> I suppose that is easy enough. 'git mv'. :)
[18:49] <jkridner> I guess I'd just want to specify the scanning as a function call that I can give arguments.
[18:49] <jkridner> I'm not a fan of boilerplate stuff. I like function calls and libraries. :)
[18:49] <zumbrunn> like midnightmonster says... move your app's code to a subdir called "code" and add a app.dir or app.repository mapping to the apps.properties file
[18:50] <jkridner> first thing I did was stop using .hac files and start calling them .js.
[18:50] <midnightmonster> it just works as it is. why make it harder? when your list of folders by design represents your list of prototype, why would you want to put other things there to confuse it?
[18:50] <jkridner> fair enough. that is easy to do.
[18:50] <jkridner> I wouldn't. It is a good way to organize things.
[18:50] <midnightmonster> you're misusing the word "boilerplate". boilerplate is all the repetitive stuff you have to type when you don't have strong conventions.
[18:51] <jkridner> fair enough. I just don't have a word in my vocabulary to describe all the "wrapper" stuff that sits outside the app.
[18:52] <zumbrunn> "conventions" :-)
[18:52] <jkridner> It isn't a huge complaint, but I'd rather own 'main' myself, and simply call a 'register project' style function.
[18:52] <jkridner> by having the framework interface be functions, rather than conventions, it is very direct to document via Doxygen/etc.
[18:52] <jkridner> jsdoc.
[18:53] <zumbrunn> but you're not forced to use them, you can script things inside your apps Global dir as you please and happily ignore everything helma provides by default
[18:53] <jkridner> you can still have something like "addApp()" or whatever that by default would have the appropriate directory structure.
[18:53] <jkridner> that's exactly what I'm working towards.
[18:54] <jkridner> Helma does so much for me automatically, it is a challenge for me to unravel it all.
[18:54] <jkridner> as I unravel it, I try to move toward my desired approach to keep things a bit more clearly documented in the code.
[18:54] <jkridner> i shouldn't even bother you with this opinion...
[18:55] <jkridner> it is just an opinion and likely only my own. :)
[18:55] <zumbrunn> no, I understand that point of view
[18:56] <zumbrunn> and I think helma should keep moving in a direction that allows that flexibility
[18:56] <zumbrunn> and makes it even easier to script around the framework
[18:56] <zumbrunn> and then just "load" the parts of the framework that are desired
[18:57] <jkridner> well, I'm due to head elsewhere. I REALLY appreciate the time. You've both been extremely helpful. If I ever do get far enough along in my app that it is interesting, you should feel free to take/give on it. The OpenID stuff is really the most interesting aspect so far.
[18:57] <jkridner> I'm glad you think that as well.
[18:57] <midnightmonster> come back and talk about that later. I've been thinking about doing openid
[18:57] <zumbrunn> yes, making the openid part a module would be interesting
[18:57] <jkridner> I've tried to shove it all under my User object.
[18:57] <midnightmonster> using any of the java libs for it or doing it all yourself?
[18:58] <jkridner> openid4java.
[18:58] <zumbrunn> that code is already in the repository you linked to a couple of days ago, right?
[18:58] <jkridner> that is another thing I need to put in my repository... all of the .jar files. :)
[18:58] <jkridner> yes.
[18:59] <jkridner> I haven't documented the .jar dependencies yet.
[19:00] <jkridner> I put all of the login stuff into http://www.beagleboard.org/gitweb/?p=beagleboard.org.git;a=blob;f=User/macros.js;h=ccbbcb624c26ee749d2e72bdafd8055b21a18344;hb=2f099ccfb830f5940fb408e14ae5ed5f918ce481
[19:00] <jkridner> I need to clean it up quite a bit.
[19:00] <jkridner> I wanted the user to be able to log in from any page, so I just made the login as macros.
[19:03] * zumbrunn wonders if the history data even needs to stay in the xml db at all
[19:04] <zumbrunn> could maybe be dumped to a lucene index instead
[19:04] <midnightmonster> why lucene? seems imprudent
[19:04] <zumbrunn> compare to what?
[19:05] <midnightmonster> lucene is "for" full-text search.
[19:05] <zumbrunn> yeah, true
[19:05] <midnightmonster> which seems like something you wouldn't be doing much on the history-as-history
[19:05] <zumbrunn> right
[19:05] <midnightmonster> and it's almost the only way to retrieve dat from lucene
[19:06] <jkridner> I don't want to leave this conversation, but I must. thanks again for your help. bye.
[19:08] <midnightmonster> adieu
[19:12] <midnightmonster> wish there was a good berkleydb-style db for java that had more liberal terms than berkleydb for java does
[19:12] <midnightmonster> (there are a whole bunch in C-land)
[19:13] <zumbrunn> what makes the berkleydb for java unliberal
[19:13] <midnightmonster> gpl
[19:13] <zumbrunn> ok
[19:13] <midnightmonster> bbl