Hopbot log for 2007-10-26 - Helma IRC channel: #helma on irc.freenode.net

2007-10-26:

[2:08] <BSlivka> Must _id be a number?
[2:22] <midnightmonster> I can't say this definitively, but I have many reasons to think yes, it must be a number
[2:31] <BSlivka> Hrmn bugger.
[2:32] <BSlivka> that puts me in a difficult spot then!
[2:33] <BSlivka> I'd hate having to add another superfluous integer column just cos helma can't concieve of a primary key being a name.
[2:33] <BSlivka> and changing the schema will be a pita
[2:35] <BSlivka> AS it is, helma keeps spitting a huge block of java errors at me.
[2:37] <BSlivka> but it doesn't quite make sense since when I echo back the autogenerated id's, there's a t at the start
[2:37] <midnightmonster> Helma does expect a meaningless integer column
[2:38] <BSlivka> how truly unfortunate. :(
[2:39] <midnightmonster> why will changing the schema be such a big deal?
[2:40] <midnightmonster> you can just change your primary key to a unique index and add an auto_increment primary key column
[2:40] <midnightmonster> the rest of everything can continue to work the same way
[2:41] <BSlivka> well first of all, I'd prefer the primary key to be what the actual primary key is- The name.
[2:42] <BSlivka> But second of all, changing the primary key in such a way will break some of the relationships in the schema
[2:42] <BSlivka> so I'd have to rebuild them from scratch
[2:42] <BSlivka> Maybe I'd get away with just adding a meaningless int column and not change any keys
[2:43] <BSlivka> But I think I will explore if there's a way to do it without touching the DB
[2:43] <BSlivka> Just briefly
[2:43] <BSlivka> in case I'm missing something obvious
[2:45] <midnightmonster> you can skip the built-in ORM and write your own mapping with HopObject.getChildElement and use DatabaseObject for your DB access.
[2:46] <midnightmonster> there are a lot of cases where you want to extend the ORM with getChildElement anyway. and if you went all crazy and used CouchDB or something, you'd need to write your own mappings completely with getChildElement.
[2:47] <BSlivka> The problem I'm having is specifically with adding new records to the database
[2:47] <BSlivka> What you suggest sounds like it would only be useful for reading from a db
[2:47] <BSlivka> Though I could be mistaken
[2:48] <BSlivka> so far, reading from the database with a non integer ID has worked flawlessly
[2:48] <BSlivka> But now I'm getting "java.sql.SQLException: The value supplied cannot be converted to BIGINT."
[2:48] <BSlivka> Which, I'm not sure exactly why it is giving this error.
[2:48] <midnightmonster> you'd have to skip using the built-in persistence, too (part of the ORM) and save your objects to the DB with sql, too.
[2:49] <BSlivka> Hrmn, that's kind of skipping most of the advantages of using helma to begin with isn't it?
[2:49] <midnightmonster> that is weird. sounds like there's a BIGINT column with an invalid value
[2:50] <midnightmonster> a lot of, not all. I'm not saying it's a good option, just the one I know of.
[2:50] <BSlivka> Sure
[2:50] <BSlivka> It's funny cos there's no BIGINT column at all in the table concerned
[2:51] <BSlivka> which makes me think there's a built in assumption in helma that an ID column is going to be BIGINT, even if it isn't
[2:51] <BSlivka> in the insertion code anyway, maybe not in the selection code, which as I said hasn't had problems
[2:52] <midnightmonster> but... why would it assume BIGINT (instead of INT)? and it would seem like that could only be thrown by attempting to save a non-int value to an existing column
[2:53] <midnightmonster> the whole stack trace should be in your log
[2:53] <BSlivka> sure.
[2:54] <BSlivka> http://helma.pastebin.com/m7bc80da3
[2:56] <BSlivka> hrmn the origin may have originated at the JDBC driver
[2:56] <BSlivka> err, the error may have originated
[3:00] <BSlivka> sure enough the error seems to be related to trying to convert varchar to bigint
[3:00] <BSlivka> on SQL SERVER 2005, via the specific driver that I have
[3:08] <midnightmonster> but I've been following the java source a bit, and it looks like that happens during Helma's attempt to generate the next ID for the table
[3:09] <midnightmonster> I think a SQLException can only be thrown from ~JDBC, so it's not surprising that you found it so. the question is what's helma doing to cause it and can it be worked around, and I don't have enough internals knowledge likely to work it out in less than an hour
[3:10] <midnightmonster> which, alas, I cannot afford to spend although it's interesting.
[3:10] <midnightmonster> you might ask on the user list--hannes can likely answer in his sleeo
[3:10] <midnightmonster> sleep
[3:12] <midnightmonster> (the trail starts at https://dev.helma.org/trac/helma/browser/helma/helma/tags/helma_1_6_0/src/helma/objectmodel/db/WrappedNodeManager.java#L227 fwiw)
[3:14] <midnightmonster> err... stupid on me
[3:15] <midnightmonster> the screwup actually happens here: https://dev.helma.org/trac/helma/browser/helma/helma/tags/helma_1_6_0/src/helma/objectmodel/db/NodeManager.java#L770
[3:16] <BSlivka> I found that if I use an Integer column for ID the error does indeed stop happening
[3:16] <BSlivka> This seems like a bit of a design problem, since it's poor practice to have database ID columns expressed in the URL space, to begin with.
[3:17] <BSlivka> (though it is common practice)
[3:17] <midnightmonster> helma asks the db for the max value of the id column. the db obliges, but when that's a string max() will be something like 'ziggy'. in line 795 helma asks the resultset to retrieve that value as a long, and crash
[3:18] <midnightmonster> but generally in helma you would not expose the database id column value
[3:18] <midnightmonster> you can map your urls on a string name column, there just has to be the int id column
[3:19] <BSlivka> from what you describe as the source of the error- it seems like it could be fixed
[3:19] <BSlivka> but I wonder what unforseen repercussions may result from fixing it
[3:19] <midnightmonster> yeah
[3:20] <BSlivka> I may just make a post about it, as you suggest
[3:23] <BSlivka> it seems like this is fundamental to the design of helma to a certain extent, because this is how the User prototype is designed
[3:23] <BSlivka> with a _name, and a redundant, _id
[3:24] <BSlivka> (since usernames must be unique to begin with)
[3:26] <midnightmonster> yes. as I understand it, having a meaningless id column is a very common if not the most common practice in db design. it would be nice if Helma could be more flexible, but to do it right would require something like python's sqlalchemy, I think, which is cool stuff but also much newer than helma's orm, which was designed several years ago
[3:28] <midnightmonster> still, the function that freaks over the varchar not converting to bigint when working correctly returns the next id value converted to a string, presumably for use in building a SQL statement. so you might be able to meet your need with little impact
[3:28] <midnightmonster> (I assume you had tried setting _id yourself before persisting these?)
[3:29] <BSlivka> Yeah
[3:29] <BSlivka> Helma doesn't let me change the value of _id
[3:29] <BSlivka> read only it would seem.
[3:29] <midnightmonster> i thought not, but that's what you'd need to [be able to] do
[3:30] <BSlivka> Helma also has _name
[3:30] <midnightmonster> read only is the correct behavior if you assume a meaningless primary key. changing it could only ever break the unique constraint and/or relations
[3:30] <BSlivka> but if you omit _id from the type.properties, it just looks for a column called ID instead, by default
[3:31] <midnightmonster> yes--_name's how you normally do the url mapping
[3:31] <BSlivka> Do you have any idea how to change _name? or is that read only as well?
[3:31] <BSlivka> I tried a setup where I have a meaningless id column,
[3:31] <BSlivka> and the _name as the primary key
[3:32] <BSlivka> but it just defaulted to the name of the prototype, and I couldn't change it
[3:32] <midnightmonster> what defaulted to the name of the prototype?
[3:32] <BSlivka> the _name property of a new object
[3:32] <BSlivka> It's either the name of hte prototype, or the name of the column
[3:33] <BSlivka> When it should be the name of the record, when I persist it.
[3:33] <midnightmonster> yeah...hang on
[3:36] <midnightmonster> your type.properties file should have:
[3:36] <midnightmonster> _name = YOUR_NAME_COL
[3:36] <midnightmonster> but if it also has
[3:36] <midnightmonster> myname = YOUR_NAME_COL
[3:36] <midnightmonster> I think maybe you can read/write myname and have it work as expected
[3:37] <midnightmonster> (the _whatever properties are all read-only and sort-of magic, I believe)
[3:38] <BSlivka> Ah
[3:38] <BSlivka> Okay I'll give that a shot.
[3:39] <BSlivka> I suppose it's not so bad- having a meaningless id column. Just kind of an unfortunate side effect.
[3:40] <midnightmonster> if you don't go for that DB theory, yeah. but like I said, many people consider meaningless primary keys best practice.
[3:40] <BSlivka> According to relational theory, the only requirement for a primary key is that it be unique
[3:40] <midnightmonster> I know
[3:41] <midnightmonster> but you're not working with a real relational db, according to relational theory
[3:41] <BSlivka> I guess the meaningless ID is a simple way of garaunteeing that
[3:41] <BSlivka> sure
[3:41] <BSlivka> I like to pretend sometimes though
[3:41] <BSlivka> You read much of Date?
[3:42] <midnightmonster> ?
[3:42] <BSlivka> Chris Date
[3:42] <midnightmonster> nope. but google tells me why I might at some point.
[3:43] <BSlivka> He and Hugh Darwen wrote a paper called the Third Manifesto, which outlines how ORM should really be done, and not run into the impedence mismatch problem
[3:44] <BSlivka> And a thick book, now in its third edition, dedicated to explaining the paper.
[3:45] <midnightmonster> it's funny b/c it doesn't surprise me in the least that that's necessary
[3:45] <BSlivka> Because the definitions of certain relational concepts have been so distorted by SQL, such things seem to be necessary.
[3:46] <midnightmonster> right... it's not so much "how to do orm" as "if anyone actually did r right, you wouldn't have so much trouble with orm"
[3:46] <BSlivka> yeah
[3:46] <BSlivka> exactly
[3:46] <BSlivka> In fact, they actually speak out against ORM, because it's a bit like type casting really
[3:47] <midnightmonster> I've read just enough to figure out that it didn't apply directly to any software I actually use
[3:47] <midnightmonster> meantime I 'm salivating over couchdb
[3:48] <BSlivka> Eh it seems kinda nice, but the way I see it, that niceness could be achieved within the relational model
[3:48] <BSlivka> and still maintain all the benefits of the relational model, such as proper reporting
[3:48] <BSlivka> logical inferrence engine
[3:48] <BSlivka> that sort of thing
[3:49] <BSlivka> which have never truly been realised
[3:49] <BSlivka> It's kind of throwing the baby out with the bathwater, I could say.
[3:49] <midnightmonster> I suspect we have very different ideas of the niceness of couchdb. but if someone makes something actually relational that does cool things, I'll be happy to check it out
[3:50] <BSlivka> It's nice that it got rid of SQL, but not so nice that it got rid of the nice things that come from set theory and predicate logic.
[3:51] <BSlivka> Eh, I've thought a lot about the relational model. It's possible that I see things in it that others don't.
[3:52] <midnightmonster> as I said, I checked out when I realized there was no actual software I could use to take advantage.
[3:52] <BSlivka> For instance, it's perfectly possible to have a query language without a single JOIN statement.
[3:52] <BSlivka> There's a sort of reference implementation, in rel
[3:52] <BSlivka> and for perl there's muldisDB
[3:52] <BSlivka> and then in C++ there's the DURO library
[3:53] <BSlivka> And on the commercial end there's Alphora
[3:53] <BSlivka> These are all properly relational
[3:53] <BSlivka> My ultimate goal is to make one that has JS as a query language akin the couchDB
[3:54] <midnightmonster> should you do so, I'll be all over it.
[3:54] <BSlivka> I'm afraid my inexperience may mean that it will suck, but I'd like to try anyway
[3:55] <BSlivka> I should probably check out CouchDB just to see what its js based query language looks like
[3:55] <midnightmonster> it doesn't
[3:55] <midnightmonster> mostly
[3:55] <BSlivka> Oh? was that just hype?
[3:56] <midnightmonster> use js to define views, then query them mostly with simple GET requests
[3:56] <midnightmonster> pass data as JSON
[3:56] <midnightmonster> (both directions)
[3:57] <BSlivka> I have an idea for a schema definition language that can generate any arbitrary level of normalization that you'd like
[3:57] <midnightmonster> (all of which looks to me like a very good thing)
[3:57] <BSlivka> yeah.
[3:57] <BSlivka> oh well. Back to work.
[3:57] <midnightmonster> word.
[7:32] <BSlivka> I'm kind of left scratching my head over this one "The _parent entry contains a comma-separated list of properties of objects of this type to be used as parent. Objects must know their parent in order to generate correct URLs in their href() function."
[7:33] <BSlivka> There are too many ambiguities, and since I don't really know what _parent is supposed to do already, I can't really suggest a correction,
[7:33] <BSlivka> except that "Objects of this type" What type? the type of the properties you're referring to, or the type that you're defining in the type.properties file
[7:35] <BSlivka> "To be used as parent" Circular, this statement is meant to be defining "parent"
[7:36] <BSlivka> "Objects must know their parent in order to generate correct URLs in their href" why is it important to generate correct urls? how do I know if I've correctly identified the "parent"
[7:36] <BSlivka> how do I know if the url's are "correct"
[7:36] <BSlivka> Perhaps a link to more information about how url's are generated?
[7:37] <BSlivka> "If an entry in the _parent item does not contain a dot, it is interpreted as an object property of the current object that acts as the object's parent"
[7:38] <BSlivka> Why would I want to do that?
[7:38] <BSlivka> On the surface it doesn't make much sense to me for an object's property (a child of sorts) to also be the same object's parent
[7:39] <BSlivka> anyways, I'm gonna check out now.
[7:39] <BSlivka> See yah

 

 

In the channel now:

Logs by date: