Suite Interoperability

[andargor]

Hey Andagor,
Looks like I made it. Thanks for the invite.

Welcome!

Andargor, I looked over your pattern and it's not bad. I think you might want to look at MVC patterns and they might help you come up with a more streamlined version of what you're looking for. I believe all of the data sources should be together like mySQL and XML files.

I'm not familiar with MVC patterns. I'll look it up.

I'm not sure why you have two data abstraction layers. One is all you need. It contains the DAO which will handle all the inserting and selecting. It gets tricky if you are using XSL to extract data and pass it to your script, because the function call comes from the script but it's using a stylesheet to do all the dirty work. So I've got two DAO classes, one for SQL and the other for XML.

Is your concern performance? My view is that it does not need to be done on the fly, but could be offline processing. Well, maybe not for character sheets, but definitely for data sets (which are probably more hefty, but do not change often).

In any case, the diagram was quickly done in about 5 minutes, and is more description of flow, rather than components.

Andargor

 

[pezagent]

Hi guys, I've been reading this thread over again (and over) and picking out bits, I can see that the priliminary discussion about "standards" usually revolves around the same universe.

I've taken the opportunity to invite the participants (who registered emails) to the new group I've started at Yahoo! groups. I started this group because I was stuck in the mud at the other group started for d-20 XML.

My main beef with these sort of things is that everybody has an idea, nobody has an application. In design, there needs to be a fairly good ratio of ideas to applications. When too many ideas get passed around the room nothing happens because action is not taken. This is true of any community input, or brainstorming scenereo. Of course, people aren't really brainstorming right now, because there's no project leader. And there's no project leader because there's no projects.

The other beef I've been having is about XML/XSL/XSD in general. People seem to think they have a firm grasp of what XML is, or what XSL can do, but in reality, I'm finding that very few people who are tossing around these ideas of creating standards aren't really aware of how these tools work. The recommendations that were put forth by the original d-20 group memebers is, IMO, totally useless, superfluous, and considers nothing about real-world applications and everything about make-believe applications. So it is imperative that anyone wishing to go beyond the idea stage not only grasp the concepts behind how XML/XSL/XSD work together, but also start using them.

Not only should one know about how the XML "system" works, one should also understand the concepts behind non-procedural programming and patterns, so that when we talk about seperating application logic from the data source people know what we're talking about. I'm not a computer scientist, I'm a progressive hack with a bit of professional experience under my belt--and a f*load of theory in my head. So I'm no expert, but I expect that others undertaking these projects should have some pattern theory so conversations don't drift into the abyss because one of the participants isn't prepped for takeoff.

Anyway, I'm going to stop there for now.

Regards,

/johnny

 

[pezagent]

Sure it is... without a common format somewhere, it means every RPG software app out there for d20 has to code up transformations for every other data format too. Essentially reinventing the wheel every time.

Hi Hollywood,

Actually, this is why XSLT was invented, and why we have XSD. Setting anything in stone is a very bad idea. No doubt I (we all) understand the problem--application A needs data from application B, how does application A get it?

The learning curve for XSL is high because it's not being explained properly, IMHO, but it's one of the most amazing things I've ever used in my experience as well as a dream come true. I wouldn't sweat the transform process--it's painless. And in some cases, totally useful to make sure there *is* a transform process. I'm trying to come up with a good example but I can't explain the goo inside my head in any easy way--the best way to summarize it all would be *mix and match*, where I can transform data A with data B, C, and D and end up with data E which can be accessed by my DAO. If there is a "common format" that is "universal" then that really defeats the purpose of XML in general. Think about it, PGGen already has a "common format" for PCGen. You can write a PHP script to parse the data written for PCGen, for example, but then you'd be locked into the PGGen way of doing things. Do you see where I'm going with this? In other words, there's no need to have a common format. Through trial and error, discussions with idiots, eggheads, professionals, geeks, and Jesus, I have discovered that as long as the XML is well-formed, an XSL file can transform it without difficulty. And that's the point. So people are free to do what they want. And when they need to exhange data, they transform it.
It's really that simple.

Now, regarding namespaces and reserved names--there's the kicker that nobody seems to be discussing short of andargor and myself. All of the game manufacturers have taken care of this for us--in the game rules. What the game rules need is specific to the game, so as long as the naming conventions match the game rules, there should be no problems.

This isn't a very difficult project. I've already completed a lot of markup that is structured in hash-tables and lists (as it is presented to us in the first place) and can easily be transformed. I'm using XSLT now to perform lookup on these tables and lists, and the code is nothing at all. It's some of the easiest code I've ever written for an application. It makes me laugh how simple this stuff is, and how complicated everyone makes due to misunderstandings.

Anyway, I'm not bashing anyone's ideas or projects, I just urge anyone undertaking these tasks to grasp the simplicity of it. Because from there, we can really start doing some totally cool sh*t with this data--and really have some fun.

/johnny

 

[DMFTodd]

I'm lurking. Not sure why Hollywood thinks my opinion is important but here it is:

I know next to nothing about XML. Programming for DM's Familiar keeps me plenty busy as is, I can't really offer much help in all this theoritical discussion. I've said I'll probably support the CSX format because it's *done*. I haven't done anything with it yet though. When I'm ready to do something with XML, I'll see what's *done* and *working* and I'll use it. I won't go roll my own.

Someone mentioned that this discussion needs to move past theory and into application. I'd agree with that. Character generation or even character sheets is very complicated. Perhaps that is not the best place to start. Maybe start with an XML format for equipment, or weapons, or spells. Something easy. If you can get that off the ground, then you can tackle the more complicated stuff.

 

[Davin]

Hi guys, I've been reading this thread over again (and over) and picking out bits, I can see that the priliminary discussion about "standards" usually revolves around the same universe.
(snip)

You seem to have some strong feelings on this subject, but I think your general complaints are valid ones. Most people doing this sort of work aren't doing (IMO) a terribly good job of generalized design. Things I've seen tend to be too task-specific, and as such they suffer badly when it comes time to communicate between disparate systems. You're going to need "indians" as well as "chiefs" in something like this, but your top-level design people will probably need to have strong coding skills (any language) and strong database design skills (any database) in addition to good technical XML-related knowledge & experience. Otherwise the design will suffer, perhaps badly.

Many people tout the brilliance of using XSL[T] to convert between formats, but they fail to take into account that most of the data structures they're converting between don't have the same data in them. To reiterate, they're each usually to specific to the task for which they were designed, so they don't tend to have the right kind of data for the application they need to be translated to. For a simple example, one application may output only "totals" for something, but the receiving application needs it broken down into details. In such a case, the translation is impossible.

I'm not sure that all these problems even have solutions, much less feasible ones. And I'm far too busy myself to do more than get my fingers in the way of the people that are really doing the work, even if I had the necessary skills to contribute profitably. But if you're serious about doing this kind of thing, you'd best do some meta-design to make sure the organization works and can produce the kind of results you're looking for.

 

[pezagent]

Hi Davin,

You seem to have some strong feelings on this subject, but I think your general complaints are valid ones. Most people doing this sort of work aren't doing (IMO) a terribly good job of generalized design.

Yes, I am passionate about this, I think because I'm dealing with some "unfinished business" I had as a kid when I first started hacking. My friends and I played D&D a lot and of course played Wizardry and all the first role-playing games available, and concepts like these (an interchangable universe) were always too difficult to organize or conceptualize for us, so they were just dreams. Now we have the technology to make this dream a reality. People can grasp the concept, but I believe the majority isn't digging deep enough to see the holistic (or meta) "top-down" approach. And if they are, they're not using the tools the way they were meant to. The result: a lot of unnecessary work, all done for nothing, that nobody is using anyway.

Things I've seen tend to be too task-specific, and as such they suffer badly when it comes time to communicate between disparate systems. You're going to need "indians" as well as "chiefs" in something like this, but your top-level design people will probably need to have strong coding skills (any language) and strong database design skills (any database) in addition to good technical XML-related knowledge & experience. Otherwise the design will suffer, perhaps badly.

Mm. When I read "task specific" I'm thinking application logic--and if that's what you're referring to I totally agree that a simple project such as isolating and marking up data seems to become this monumental undertaking for people. It's like, slow down people, one step at a time.

For me, the projects I'm interested in and the work I'm doing, people need to know the fundamental approach to XML, why documents get transformed, and, especially for a vast gaming system, the concept of namespaces using XSD. People are still trying to construct DTDs, which are totally useless for the scope of any project involving so many resources.

And you're correct in saying that the design will suffer. I think most people who joined the d20-XML group became frustrated with the lack of knowledge, support, and constant debate over trivial issues. I know I did. And the results of that organizational structure, that group, are clear: there really aren't any results. Chalk it up to one project, too many chiefs, not enough indians. Absolutely.

Many people tout the brilliance of using XSL[T] to convert between formats, but they fail to take into account that most of the data structures they're converting between don't have the same data in them. To reiterate, they're each usually to specific to the task for which they were designed, so they don't tend to have the right kind of data for the application they need to be translated to. For a simple example, one application may output only "totals" for something, but the receiving application needs it broken down into details. In such a case, the translation is impossible.

Well, an application certainly has to take on some responsibility for itself. And it's important not to leave out the most important components of the application--those using it and those designing it. I think those sort of problems can be resolved easily with feedback. The beauty of XML is that it is extensible, and new elements can be added on the fly without too much complication. Of course, this is up to the application developer, but as it has been proven historically, applications that don't meet the demands of users become ex applications. There's plenty of room for development. Anybody can build a better mousetrap.

I'm not sure that all these problems even have solutions, much less feasible ones. And I'm far too busy myself to do more than get my fingers in the way of the people that are really doing the work, even if I had the necessary skills to contribute profitably. But if you're serious about doing this kind of thing, you'd best do some meta-design to make sure the organization works and can produce the kind of results you're looking for.

Thanks for the encouragement. I have taken steps towards a better future for data exchange between d20 systems, and one of them was to create a new group, the d20-dataExchange, which most people in this thread have already been invited to participate in. And certainly anyone reading this thread with a genuine interest to pursue dataExchange between systems is welcome to take a look and see if this group is what you're looking for.

Best regards and thanks for the thoughtful reply,

/johnny

 

[Davin]

Mm. When I read "task specific" I'm thinking application logic--and if that's what you're referring to I totally agree that a simple project such as isolating and marking up data seems to become this monumental undertaking for people.

Actually, what I meant had to do with the data people were using. For instance, if application X tracked (and exported) detail data A, B, and C, but application Y only handled the total of A+B+C, then you could transfer from X to Y, but not back from Y to X (because Y was exporting insufficient information because that's all application Y wanted to deal with).

Best regards and thanks for the thoughtful reply,

You're quite welcome.

 

[pezagent]

Actually, what I meant had to do with the data people were using. For instance, if application X tracked (and exported) detail data A, B, and C, but application Y only handled the total of A+B+C, then you could transfer from X to Y, but not back from Y to X (because Y was exporting insufficient information because that's all application Y wanted to deal with).

Hi Davin,

Well, this explanation is clear, thank you. I agree with your point, however, I'd like to look at the problem from a less ambiguous point of view. I've mentioned before that I believe application development has to meet the needs of its users. I don't think anybody's going to get it right on the first try. Feedback, clear communication, and goal-setting will help cross those bridges when they are reached.

/johnny

 

[Hollywood]

Hi Hollywood,

Actually, this is why XSLT was invented, and why we have XSD. Setting anything in stone is a very bad idea. No doubt I (we all) understand the problem--application A needs data from application B, how does application A get it?

XSLT is nothing but a new acronym on an old subject. Much as HTML, XML, VML, etc. are all markup language derivatives. Markup languages have been around for over two decades. Nothing new really here.

And while, yes, XSL style sheets allow you to transform XML markups, its rather pointless to transform XML from one format to another format if the formats contain roughly the same information and have a very close data structure. Might as well simply use the same common format and skip the need for doing a transformation all together. The XSLT process is not exactly the most efficient thing in the world... although in small datasets it can appear to be quite efficient.

And since its so easy, let's see your schema or DTD and/or your application that uses it.

 

[pezagent]

XSLT is nothing but a new acronym on an old subject. Much as HTML, XML, VML, etc. are all markup language derivatives. Markup languages have been around for over two decades. Nothing new really here.

Which "old subject" are you referring to here? XSL is a declaritive programming language, currently evolving, which uses other languages such as XPath. SGML, the parent of all markup, is as old as I am, and that's a bit more than two decades. Try three and a half.

It would be interesting to see you try and run that statement by the Mulberry list and see what sort of response you get there--if anyone would actually bother to respond to it.

And while, yes, XSL style sheets allow you to transform XML markups, its rather pointless to transform XML from one format to another format if the formats contain roughly the same information and have a very close data structure. Might as well simply use the same common format and skip the need for doing a transformation all together. The XSLT process is not exactly the most efficient thing in the world... although in small datasets it can appear to be quite efficient.

I'm not sure what you mean here by "pointless." And when it comes to computers I'm not sure I like the idea of "roughly the same information," or "very close data structure." There's no room for approximations when developing applications. We're not rearranging furniture or putting up curtains here. Different applications require different data structures. The application isn't built around the data format.

Transformation allows two applications to retain control over a proprietary data structure and easily share that information with other applications without others having to write a new parser for each transformation. Furthermore, XSL not only transforms the data but can parse it, merge it, sort it, group it--most anything you need to be done to the data can all be done with stylesheets.

The whole point of transformation is so XML documents don't have to have a common, or "standard" format. So application developers can build data structures and change them later as needs dictate. That's the whole point of using XML as an extensible markup language.


/johnny