JavaScript, jQuery, and UI Frameworking

In my professional consulting career, I’ve been avoiding the UI area of software development for years, much like I tend to avoid reporting tools like Crystal Reports or Business Objects. I like poking around in the guts of systems. Everything below the surface is so much more interesting, or so I thought. Instead of working on UI layers, I tried to hire other people to do what I thought was fairly simple work. This wasn’t working. It turns out, in order to direct hired hands, you need to know what the hell you’re asking them to do. So I backed off having other people do UI work and realized I just need to strap in and learn it on my own. This includes iOS, JavaScript, jQuery, and anything else to do with building user interfaces. It turns out, there’s a lot to learn.

I’ll talk about iOS in a future blog entry, once I get comfortable with iOS, Objective C, and all the related tools. Today I’m going to talk about JavaScript and jQuery.

In a side-project called Zifmia, I’m developing a service/cloud based game engine for Interactive Fiction. I’ve been working on this in pieces for a long time. It’s morphed from WPF, to Silverlight, to Java, to iOS, and I finally realized that if I could create a service, I don’t need to port it anywhere anymore. Well, except maybe the Kindle since the Kindle doesn’t support connected services. (This makes sense. Most people use the Kindle for reading and are only concerned about connectivity when purchasing and downloading a new book or other content.) But for the web, smartphones, the iPad, iPod, and other tablets, a service/cloud engine is really the right way to go.

I start developing the service layer using RESTful web services. I used the Microsoft .NET WCF REST 4.0 template to provide the external layer for the service side functionality. This was great since WCF 4.0 REST works nearly perfectly out of the box. There’s very little “tuning” to do to make it work locally or on a server. The one major issue I had was with cross-domain ajax calls, but that’s solved by adding a header on all return calls from the server. The service allows a consuming application to register a user, login, start a session (game), send commands to a session, and various other management tasks. I plan to implement oAuth so that the service can be embedded in Facebook properly.

I then start playing with various platforms to consume the service. I played round with Windows Phone 7 (Silverlight) and that was drop dead simple. It takes an object definition and two lines of code to consume the service. I didn’t implement a full Application, but it helped work through some of the service prototyping issues.

I played around with iOS, but stalled out because I wanted the web interface completed first. I had a very committed developer working on it, but our communication wasn’t working well, and so I set that relationship aside for the time being. (I haven’t forgotten about you Dan…hope to catch up soon). I then started to work on the client side JavaScript code and realized there was a LOT of work that needed to be done before anyone could use Zifmia properly. The result of that work is the essence of this article.

I started developing the interfaces in jQuery and JavaScript, but soon realized the code (HTML, CSS, JavaScript, jQuery) was becoming quite convoluted and unreadable. I also realized it would only get worse if I continued. So I started asking around and researching how JavaScript/jQuery based client user interfaces should be organized. The surprising answer is that although there are a handful of serious solutions, there isn’t a lot of open thinking about how this should be done. The people that understand the need and have the ability to work through the problems have created tools, but haven’t really shared the process with the public. It’s a shame, since that’s the really important part to share, not just the code.

I looked at a few of these tools (like JavascriptMVC, Backbone.js, SproutCore) but found all of them to have aspects I dislike. Complexity, dependencies, and high learning curves. Instead of adopting, I chose to develop my own, trying to work through the issues. Here is what I came up with. This is a very simple methodology for separation of concerns in a javascript UI framework.

The first order of business is to identify assumptions:

• We will work on the concept of a single page interface.
• The page will have its own class/namespace.
• The page will have a master HTML arrangement (header, body, footer).
• The body will be a View container.
• Each view will be based on an HTML template separate from the main html page. The HTML template will use normal HTML, CSS, and JavaScript.
• Each View will have its own Controller or Presenter class.
• All AJAX calls will be contained in a single Service class. (This is arguable, since a more complex web application could have highly distinct groups of service calls. More than one Service class could be adopted).

The result is a very simple and effective way to separate code in a single page framework jQuery driven website.

Create a web page called index.html similar to the one below:

<html>
<head>
    <title>Zifmia></title>
    ....
</head>
<body onload="javascript:$MyWeb.onLoad();">
    <div id="outerPanel">
        <div id="ajaxLoading"><img src="images/loading.gif" alt="Loading" /></div>
        <div id="headerPanel"><p>Textfyre Presents, A Zifmia+Web Portal (Beta)</p></div>
        <div id="viewContainer"></div>
        <div id="footerPanel">Copyright @ 2011 Textfyre, Inc. - All Rights Reserved</div>
    </div>
</body>
</html>

In the head section of the html page, create a class and create an instance of the class as MyWeb:

var $MyWeb = new mySite();

function mySite() {

    this.controller = null;

    this.username = getCookie("username");

    this.onLoad() = function() {
        // Initialize page here.

        // based on your page logic, cookies, etc, determine the initial state of the page
        // this is where you would determine which view to show, or to show the default
        // view. Based on whatever view is needed, we set that controller to the page
        // controller and call init(), which should be a standard function on all
        // controllers.

        if (username != "") {
            this.controller = new sessionController();
        } else {
            this.controller = new defaultController();
        }

        this.controller.init();
    };

}

Since I assume you have a design in mind, you should be able to separate the various views on your own into html templates. Each template might look something like this:

<div id="loginPanel">
    <div style="font-size: 14pt; font-weight: bold;">
        Zifmia Login Form<span id="loginMessage" style="font-size: 10pt; font-weight: normal;
            margin-left: 20px;"></span></div>
    <hr />
    <table>
        <tr>
            <td>
                <span style="white-space: nowrap;">
                    Username:
                    <input class="login" id="zLogUsername" type="text" onkeyup="javascript:validateRequiredField(this);" /><span
                    id="zLogUsernameError"></span></span>
                <span id="nickNameField" style="margin-right:10px;font-weight:bold;">
                </span>
            </td>
            <td style="white-space: nowrap;">
                Password:
                <input class="login" id="zLogPassword" type="password" onkeyup="javascript:validateRequiredField(this);" /><span
                    id="zLogPasswordError"></span>
            </td>
            <td style="white-space: nowrap;">
                <a id="zLogSubmit" href="#login" onclick="javascript:$MyWeb.controller.login();return false;">login</a>
            </td>
            <td style="text-align: right; padding-left: 20px;">
                <span id="registerNotice">If you don't have an account, please <a href="#showRegistration"
                    onclick="javascript:$MyWeb.controller.showRegistration();return false;">register</a>.</span>
            </td>
        </tr>
    </table>
</div>

This html should be stored in a subdirectory called templates with the name loginTemplate.html. We can use a simple AJAX call to retrieve it:

    var loginTemplate = getTemplate("templates/loginTemplate.html");

    function getTemplate(templateURI) {
        return $.ajax({
            url: templateURI,
            global: false,
            type: "GET",
            async: false
        }).responseText;
    }

The getTemplate function should be placed in a utility class or in the page controller for easy reuse. From here’s simple to create a controller class to display a view template in the correct place in our html page:

function introController() {

    var REG_USERNAME_FIELD = "zRegUsername";
    var REG_PASSWORD_FIELD = "zRegPassword";
    var REG_NICKNAME_FIELD = "zRegNickname";
    var REG_EMAIL_ADDRESS_FIELD = "zRegEmailAddress";

    var LOG_USERNAME_FIELD = "zLogUsername";
    var LOG_PASSWORD_FIELD = "zLogPassword";

    var viewContainer = "#viewContainer";
    var usernameField = "#zLogUsername";
    var nicknameField = "#nickNameField";
    var regErrorPanel = "#regErrorPanel";
    var loginMessage = "#loginMessage";

    this.introTemplate = $Z.getTemplate("templates/introTemplate.html");
    this.loginTemplate = $Z.getTemplate("templates/loginTemplate.html");
    this.registrationTemplate = $Z.getTemplate("templates/registrationTemplate.html");

    this.init = function (message) {

        $(viewContainer).hide();
        $(viewContainer).html($(this.loginTemplate).html());

        var zLogUsername = document.getElementById(LOG_USERNAME_FIELD);
        var zLogPassword = document.getElementById(LOG_PASSWORD_FIELD);

        setRequiredValidator(zLogPassword);

        if (message != null && message != "") {
            $(loginMessage).text("(" + message + ")");
        } else {
            $(loginMessage).text("");
        }

        if ($Z.username != "") {
            //
            // We know the username and nickname, but the user is logged out.
            //
            $(usernameField).hide();
            zLogUsername.isValid = true;
            $(nicknameField).show();
            if ($Z.nickname != "") {
                $(nickNameField).html('<a href="#changeUser" onclick="javascript:$MyWeb.controller.changeUser();">' + $Z.nickname + '</a>');
            } else {
                $(nickNameField).html('<a href="#changeUser" onclick="javascript:$MyWeb.controller.changeUser();">' + $Z.username + '</a>');
            }
        } else {
            //
            // We don't know anything about the user.
            //
            setRequiredValidator(zLogUsername);
        }

        $(viewContainer).append($(this.introTemplate).html());
        $(viewContainer).show();

    };

    this.changeUser = function () {
        $Z.username = "";
        $Z.saveSessionData();
        this.init();
    };

    this.login = function () {
        var zLogUsername = document.getElementById(LOG_USERNAME_FIELD);
        var zLogPassword = document.getElementById(LOG_PASSWORD_FIELD);

        $Z.login(zLogUsername.value, zLogPassword.value,
                function (zifmiaLoginViewModel) {
                    if (zifmiaLoginViewModel.Status == "0") {
                        $Z.username = zRegUsername.value;
                        $Z.authKey = zifmiaLoginViewModel.AuthKey;
                        $Z.nickname = zifmiaLoginViewModel.Nickname;
                        $Z.saveSessionData();
                        $ZWeb.onLoad();
                    } else {
                        $(regErrorPanel).text(zifmiaLoginViewModel.Message);
                        $(zLogUsername).focus();
                    }
                },
                function (xhr, textStatus, errorThrown) {
                    $(regErrorPanel).html(xhr.responseText);
                }
            );
    };

    this.showRegistration = function () {
        $(viewContainer).hide();
        $(viewContainer).html($(this.registrationTemplate).html());

        var zRegUsername = document.getElementById(REG_USERNAME_FIELD);
        var zRegPassword = document.getElementById(REG_PASSWORD_FIELD);
        var zRegNickname = document.getElementById(REG_NICKNAME_FIELD);
        var zRegEmailAddress = document.getElementById(REG_EMAIL_ADDRESS_FIELD);

        setRequiredValidator(zRegUsername);
        setRequiredValidator(zRegPassword);
        setRequiredValidator(zRegNickname);
        setRequiredValidator(zRegEmailAddress);

        $(viewContainer).show();
    };

    this.register = function () {
        $(regErrorPanel).text("");
        var zRegUsername = document.getElementById(REG_USERNAME_FIELD);
        var zRegPassword = document.getElementById(REG_PASSWORD_FIELD);
        var zRegNickname = document.getElementById(REG_NICKNAME_FIELD);
        var zRegEmailAddress = document.getElementById(REG_EMAIL_ADDRESS_FIELD);

        $Z.register(zRegUsername.value, zRegPassword.value, zRegNickname.value, zRegEmailAddress.value,
                function (zifmiaRegistrationViewModel) {
                    if (zifmiaRegistrationViewModel.Status == "0") {
                        $Z.username = zRegUsername.value;
                        $Z.nickname = zRegNickname.value;
                        $Z.saveSessionData();
                        $ZWeb.controller.init(zifmiaRegistrationViewModel.Message);
                    } else {
                        $(regErrorPanel).text(zifmiaRegistrationViewModel.Message);
                        $(zRegUsername).focus();
                    }
                },
                function (xhr, textStatus, errorThrown) {
                    $(regErrorPanel).html(xhr.responseText);
                }
            );
    };
}

Note there are many other things happening in this controller including ajax calls and handling user interactions. The view is shown to the user through a jQuery call (which can be done with simple DOM manipulation):

        $(viewContainer).hide();
        $(viewContainer).html($(this.loginTemplate).html());
        $(viewContainer).show();

So that’s it. You have a main controller in your index.html page that can be access by using $MyWeb at any time, you have varied controllers that are at $MyWeb.controller that have methods that can be called from your html. All of your HTML is in separate files for easy editing and reuse. This is a simple, yet effective way to separate and manage your AJAX/jQuery/JavaScript enabled website.

Object Oriented Databases

While working on a side project, I had a thought to test an object-oriented database. My requirements are very narrowly focused and I’m in complete control of the environment, so I thought this would make a good test-subject.

I looked around and found the comparison of oo databases on wikipedia. Since I’m working in C#/.NET, I need an oodb that supports .NET well. I also prefer to find something with a community or start-up license, since I can’t afford to spend money on this. I also talked to a colleague, Jeff Panici, who recommended looking at Poet.

I first tried DB4O, since it had a demo available. The licensing requirements were a bit vague, but I thought I could work through the issue. I implemented a new database class in my Zifmia Service using DB4O and was successful at the first line of unit tests, but as I got into the deeper aspects of my object graph, DB4O seemed to act strangely. Then I inquired about the licensing issue and surprisingly, Versant expects royalties. I assume this is for mobile implementations, but apparently it’s across the board.

I will say I got the OODB bug bad. Using DB4O was light years easier than using SQL Server and managing the mappings between relational tables and objects. Just removing the need to think about relations and tables saved me an enormous amount of effort. I was determined to find an OODB solution for Zifmia at this point.

I then looked for Poet and discovered that Versant had purchase Poet and has it “hidden” from testing, use, or even purchase. I assume Poet was a legitimate competitor to their main OODB and so they bought it and took it off the shelf. This left me feeling like Versant wasn’t a company I wanted to deal with, so I went back to the comparison list.

The next OODB I looked into was Eloquera. This had everything I needed and even better, it was an almost exact match in usage patterns to DB4O. I had to update my code a little bit, but most of the Linq queries I implemented remained unchanged. I have since been able to implement all of my unit tests with success. There is one issue though. the performance is less than ideal. It’s not terrible, but it is suspect. With a very simple object graph and what is essentially 13 objects in a 3-level graph with a couple of circular references, saves are taking 3 to 5 seconds. Loads are <1 second, but the saves are a concern. It’s also slow handling large byte arrays (7MB). SQL Server was never a bottleneck for any of these interactions, but the I was doing all the mapping with SQL Server.

I’m completely in love with the OODB concept and Eloquera is great, but I’m hoping I can work with the development team to get past the performance issues.

That said, I wonder why Microsoft doesn’t create an OODB. They certainly have the know-how. It may confuse their SQL Server customers, but it may also provide an alternate solution to problems that are founded on objects, not on relations. I suspect that anyone at MS Research that’s interested in OODB development is quietly asked to choose another subject-matter. SQL Server is probably one of the golden geese and tampering with its identity at MS is likely taboo.

Follow-up: I found a 6 year old MSDN article that claims OODB’s are dead. An interesting and incorrect notion. The argument is that companies will never abandon the relational model and the problems in managing object graph changes is too complex. I’ve managed the upgrades of relational models and let me tell you that’s no picnic either. I do understand how a larger company wouldn’t want their payroll or invoice systems in an OODB. There’s too much gravity in relational database systems backing those types of vertical products. But that still leaves a ton of room for single purpose systems, mobile applications, and small business systems. These types of implementations aren’t “corporate” and can be adapted to the OODB concept. And if there’s ever a need to change the objects, it doesn’t get any simpler to write data movement Linq statements to do that work. Given two different object models, I could write a mapping program in very little time. Doing the same for a relational database would take much longer, even using tools like SSIS.

I think the MSDN artical was seriously mistaken in its grasp of how much more effcient an OODB can be.

Why iOS is King and How the King might Fall

As a start-up owner focusing on mobile game development, I’ve been staring at all of the platforms and trying to identify why Apple has been so successful and why others, including Android might struggle against them. I have a theory and want to share it.

Apple has three devices. The iPod Touch, the iPhone, and the iPad. They cover different segments of ownership. The iPod Touch has a fairly wide demographic, all the way from elementary school kids to adults. The iPhone is more of an adult toy, but parents probably share the device with their kids for the apps (Doodle Jump, Angry Birds). The iPad also has a wide range of users, but probably leans a bit towards the younger age group. No matter what though, Apple has a set of devices that covers all demographics.

This cannot be said for Android, Blackberry, Windows Phone 7, WebOS or any other device stratgey. However there are signs that the competition understands the problem.

Samsung has introduced the GalaxyTab, Microsoft has the Zune, and Archos has some smaller footprint tablets.

I think what any given vendor needs to do is focus on a set of devices similar to the Touch, iPhone, and iPad and make them seamlessly work exactly the same.

Take Microsoft. If they were to adapt the Zune to use the WP7 OS sans the phone capabilities and sell it competitively, this could be seen as a counter to the iPod Touch. If they were to also adapt a tablet to use the WP7 OS, this could counter the iPad. Microsoft could very quickly have the same device strategy as Apple and be able to compete on developer base. C# and Silverlight vs. Objective C and Cocoa? Interesting battle.

On the Android side, it seems the audience for their devices are different than Apple. They aren’t buying devices to play Doodle Jump or Angry Birds, but to locate restaurants and surf the web. I wonder if anyone will create an iPod Touch-like device that runs Android. Could be a winning stratgey. An iPod Touch for $79 instead of $149? Might be interesting.

Moving Files from XAP to Isolated Storage for Local HTML Content on Windows Phone 7

For the past few months I’ve been working on porting The Shadow in the Cathedral to Windows Phone 7. Chris Cavanagh has been doing the bulk of the UI framework implementation while I’ve been working on adding all of the extras like help, hints, and other “nice touches”.

Adding documentation to a WP7 app is tricky. You can use PDF’s, but that seems shoddy to me. You can lay out tons of pages of large text for the user to page through in XAML, but this is painful and you don’t get the multi-touch and zoom capabilities by default.

Then you discover the WebBrowser control and you think you’re home free. Just make some web pages for your documentation and plug them into the phone application, right? Wrong. It’s not that simple quite yet. WP7′s SDK is still a work in progress and only a ton of usage will refine its design to a point where developers can do most things easily.

The WebBrowser control is currently meant for reaching the Internet. It has the capability of serving local content, but only from IsolatedStorage, not from content or resources in your XAP file. At least not yet. I’m not sure what the reasoning is behind this, but it is what it is.

So in order to serve your fancy html documentation, you need to move it to IsolatedStorage from your XAP file sometime when your application executes. I spent a little time this past week developing a couple of extension methods to help do exactly this.

    public static class ISExtensions
    {
        public static void CopyTextFile(this IsolatedStorageFile isf, string filename, bool replace = false)
        {
            if (!isf.FileExists(filename) || replace == true)
            {
                StreamReader stream = new StreamReader(TitleContainer.OpenStream(filename));
                IsolatedStorageFileStream outFile = isf.CreateFile(filename);

                string fileAsString = stream.ReadToEnd();
                byte[] fileBytes = System.Text.Encoding.UTF8.GetBytes(fileAsString);

                outFile.Write(fileBytes, 0, fileBytes.Length);

                stream.Close();
                outFile.Close();
            }
        }

        public static void CopyBinaryFile(this IsolatedStorageFile isf, string filename, bool replace = false)
        {
            if (!isf.FileExists(filename) || replace == true)
            {
                BinaryReader fileReader = new BinaryReader(TitleContainer.OpenStream(filename));
                IsolatedStorageFileStream outFile = isf.CreateFile(filename);

                bool eof = false;
                long fileLength = fileReader.BaseStream.Length;
                int writeLength = 512;
                while (!eof)
                {
                    if (fileLength < 512)
                    {
                        writeLength = Convert.ToInt32(fileLength);
                        outFile.Write(fileReader.ReadBytes(writeLength), 0, writeLength);
                    }
                    else
                    {
                        outFile.Write(fileReader.ReadBytes(writeLength), 0, writeLength);
                    }

                    fileLength = fileLength - 512;

                    if (fileLength <= 0) eof = true;
                }
                fileReader.Close();
                outFile.Close();
            }

        }
    }

The next step is to add all of your content (html, images, etc) to your project and set the Build Action on every file to Content.

Then execute steps to create your directories and copy your files.

            storageFile.CreateDirectory("HTDocs\\images");
            storageFile.CopyTextFile("HTDocs\\index.html", true);
            storageFile.CopyTextFile("HTDocs\\about.html", true);
            storageFile.CopyTextFile("HTDocs\\games.html", true);
            storageFile.CopyTextFile("HTDocs\\support.html", true);
            storageFile.CopyBinaryFile("HTDocs\\images\\image1.png", true);
            storageFile.CopyBinaryFile("HTDocs\\images\\image2.png", true);
            storageFile.CopyBinaryFile("HTDocs\\images\\image3.png", true);
            storageFile.CopyBinaryFile("HTDocs\\images\\image4.png", true);
            storageFile.CopyBinaryFile("HTDocs\\images\\image5.jpg", true);

The first command “CreateDirectory” is one of the existing methods in the IsolatedStorageFile object.

The CopyTextFile and CopyBinaryFile are the extension methods I’ve added to simplify moving Content files to IS.

In order to serve your content in a WP7 WebBrowser control, you simply set the Base and call the first page.

        private void webBrowser1_Loaded(object sender, RoutedEventArgs e)
        {
            webBrowser1.Base = "HTDocs";
            webBrowser1.Navigate(new Uri("index.html", UriKind.Relative));
        }

In your html, images and other files are relative to the Base. I have not tried JavaScript or CSS files, but I assume they will work the same as images.

In order to make your HTML compliant with the WP7 browser, make sure you have a DOCTYPE tag.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

Also make sure you set the viewport in a meta tag, otherwise your HTML will shrink. I think the default viewport is 1000px wide and it’s just easier to set it to 460px wide and work from there. Make sure you manage your margins too.

    <meta name="viewport" content="width=460, user-scalable=no"/>

Now you have everything you need to develop HTML based local content in your Windows Phone 7 application.

Programming in Grades

(this post is a work in progress, but I’m sharing it now to think about the grading more)

As an architect and a developer I have often been frustrated by the gap in understanding between what I call “rock star” programmers and “average joe” developers. To start, let’s define who’s who:

Rock Star Developer
The rock star developer is the guy that eats code in all forms at any level of complexity. He can read it, decompile it, reconstruct it, and communicate it to others. What he may not be able to do is understand the big picture and how code relates to a business. If he does, then he’s a developer God and there are an infinitely small number of those people.

Average Joe Developer
The average joe developer is the guy that writes code for a living, but would very likely rather be doing something else. He has a life that doesn’t include much time on the computer beyond his work day. He’s not going to seek new knowledge. He only grows when internal systems and standards change and he’s forced to learn new things, which he will do happily, but not ambitously.

Having been in this business for 25 years (many as a consultant), I have witnessed many IT environments. It’s my belief that the number of Rock Star developers is a relatively small one. By far, the vast majority of developers are Average Joe’s.

This presents a very serious problem in most IT departments in that the minority is in control of the standards and architecture for the majority. I can’t tell you how many times I’ve witnessed the development of a “framework” that is great on paper, sells well to the business, but no one ever asks the Average Joe’s what they think. Everyone just expects that they will learn it and move on.

The problem is that this will stress out most Average Joe’s and they will become less productive. If they can’t grasp the new “framework”, they’re likely to take months if not an entire year to learn the environment.

So then you’ll hear the rock star’s say, “This isn’t that hard. All good developers should know this stuff.” and will convince management to hire consultants or more senior developers. This creates an imbalance where too many cooks are in the kitchen and egos will start to come into play. IT departments are no different than a carefully developed fish tank and its ecosystem. If you have too many rock starts, things will get unmanagable. If you don’t have any, standards will be tossed aside and you’ll have as many styles as you do programmers.

So there are a couple of problems here. One is to balance out the number of rock star developers with average joe developers. The other is, how do you determine who’s a rock star and who isn’t. So I’ve come up with a grading system to define what I think they are and you can chime in with your thoughts. We’re going to categorize various development tasks with a grade level (as in 1st grade, 12th grade, associates degree, bachelor’s degree, masters, doctorate. We’ll keep the basics at 4th grade. These are the most basic tasks a programmer needs to be productive. The scope of this grading system is strictly business application development. There are other areas of development, such as programming computer chips, that are so different that they need their own grading system. This also ignores communication skills like speaking/writing English.

4th Grade Developer Tasks

• G4-01: Ability to use a Windows, Linux, or Apple OS X computer, including log in, run programs, use a command line, print documents, e-mail, instant message, and use a text editor.
• G4-02:Ability to use the file system to locate, list, and manage files.
• G4-03:Ability to write a Hello, World! program in at least one programming language without a book.

6th Grade Development Tasks

• G6-01: Ability to use a platform specific Integrated Development Environment such as Visual Studio or Eclipse.
• G6-02: Ability to create a basic web page with a form that posts data to a server.
• G6-03: Ability to handle server side logic of a given web page in at least one programming langauge.
• G6-04: Ability to handle querystrings and form variables.
• G6-05: Ability to access a database using at least one programming language.
• G6-06: Ability to use data from a database to output to a web page.
• G6-07: Ability to write simple CRUD SQL statements.

8th Grade Development Tasks

• G8-01: Ability to create a class with private and public members.
• G8-02: Ability to create SQL Statements with one JOIN.
• G8-03: Ability to create Stored Procedures and Views.
• G8-04: Ability to write a Functional Requirements document.

10th Grade Development Tasks

• G10-01: Ability to create protected and static members on a class.
• G10-02: Ability to create an abstract or base class with virtual members.
• G10-03: Ability to create a sub class with overriding members.
• G10-04: Ability to develop a complete CRUD application in at least one programming language, including the data model, the object model, a data access layer, screens, web pages, with validation, exception handling, logging,
• G10-05: Ability to create SQL statements with multiple JOIN’s.
• G10-06: Ability to create database Functions, Indexes, and constraints.
• G10-07: Ability to develop a simple data access layer.
• G10-08: Ability to designa a simple domain model.
• G10-09: Ability to design a simple data model.
• G10-10: Ability to write a Detail Design Document.
• G10-11: Ability to write a unit test class.

12th Grade Development Tasks

• G12-01: Ability to develop polymorphic class structures.
• G12-02: Ability to develop custom generic types.
• G12-03: Ability to diagnose database performance issues.
• G12-04: Ability to design a complex domain model.
• G12-05: Ability to design a complex data model.
• G12-06: Ability to develop by writing unit tests first.

BS Level Development Tasks

• GBS-01: Ability to create and use complex data structures.
• GBS-02: Understand and use big O notation.

Masters Level Development Tasks

• GM-01: Ability to develop a compiler for an existing programming language.
• GM-02: Ability to develop a compiler for a new programming language.

PHD Level Development Tasks

Architecting From the Bottom Up

I’m sure I’m going to steal someone’s cheese with this one, but I have very strong feelings about how systems are developed and who is involved in the process.

It’s been my experience that most companies employ someone in the Chief Architect role. This person interacts with the stakeholders to determine short and long-term technical needs, works with managers to integrate architectural change through internal product development, and above all else, let no man enter the Ivory Tower.

For whatever reason, many really smart (I mean really really smart) developers make their way into the role of Chief Architect without ever understanding who their audience is. They assume their audience is the IT Director, the Board, the principals, and all managers. It’s my contention that this is entirely wrong. The main audience for architecture is the developer. And not just any developer. I’m talking about Joe Average Developer. I’m not talking about the clock-watching consultant who tries to get his straight-eights in. I’m not talking about the workaholic senior developer who, if he (or her – no gender slights meant) doesn’t understand something, will draw his own blood in nightly rituals of self-abuse in order to gain an understanding. I’m talking about the developer that comes to work, gets handed an assignment, and wants to do a good job. They know they’re not likely to be a senior developer. They’re probably going to fork off into business analysis or project management at some point. But for now, they’re a developer. Heck, they may actually be an actor or singer who happens to know how to code. Coding pays the rent.

Back to our architect. The architect also assumes that they know what’s best for the company. They may claim they have no prejudice but believe me, they do. We all do. Out of the gate, all architects are suspect zero. We all have an agenda, we all like things a certain way. In most cases, the architect will arrange the pieces on the board that makes us feel the most comfortable. We will set out on a relentless quest to see a technical environment come to life that bears our heart and soul. We do this because we know that we can be successful in such a world. Well, actually success isn’t the motivator. That’s an illusion. We want to control our environment. That is our primary goal and in many cases, the architect doesn’t even understand their own primary goal.

However, there is another way. I see the lead architect role with four primary responsibilities.

First, there has to be strong communication with all levels. The architect should know the stakeholders, the analysts, the developers, the testers, the infrastructure staff, and the users. They should have strong communication networks with all of these people (and probably more). If you have an architect that likes to hide, boy do you have problems. I’m not talking about someone who’s digging in to get work done. I’m talking about someone that you’re never quite sure what they’re doing.

Second, there has to be a strong set of standards written down, introduced at all levels, discussed at all levels, and monitored at all levels. This may include code reviews, static code analysis, check-in policies. Developers like the freedom to write code, but they don’t like everyone doing it differently. In this respect, developers are a lot like soldiers. If you give them strict guidelines, they will find the art of their skill in being productive. They will be better for it and so will your company. If standards are given short-shrift or treated as an aside, your developers will be unhappy, less-productive, and very likely to switch jobs after the next disappointing pay-raise.

Third, all new technical requirements should be developed with all hands on deck. That means that developers should have a say in any newly developed framework. Everyone should be given the opportunity to provide technical ideas and in fact, the entire process should be presented in such a way that developers are encouraged and rewarded for their input. The less an architect listens to the developers, the more costly and time-consuming any and all new projects will become.

Fourth, and this is probably the most important. Keep it simple. I’ll say it again. Keep it simple. I don’t care if you have three PhD’s in math, engineering, or computer science. You are not going to be writing code. The developers, Joe Average Developer, are going to be writing code. So if your plan is to implement a complex set of requirements as a framework, you have failed before one line of code is written in your new framework. If you have an architect that constantly talks about object-relation mapping, object containers, dependency injection, separation of concerns, and abstraction layers, you have an architect that will create a set of middleware that no one, and I mean no one, will understand. If he gets hit by a bus, you’re screwed.

I have a rule. It’s a pretty simple rule. Take any framework or middleware or API. Grab any Joe Average Developer off the street and ask him or her to create a small data entry application with three or four related tables or a handful of related classes. It doesn’t matter if they start at the domain level or the database level. They get the same assistance that anyone else does on staff. If there’s training or pair programming in the regimen, that’s great. Here’s the rule. If Joe Average Developer can’t deliver a simple application in less than two weeks (and that’s a very forgiving schedule in my book), then you have completely failed at creating a productive and cost-effective framework. If you can’t get them to create a screen or report or web page in a few days, you’ve really messed up.

Some of the arguments I’ve heard over the years include:

“We know there are average developers, but we’re going to segregate them off into simpler jobs, like UI development.” – This is great for new development. What happens when someone has to maintain the code and all the smart people are missing?

“This isn’t that hard. I find it hard to believe that a good programmer won’t understand the framework.” – This is hubris. Don’t smart people know they’re smart? I don’t like to be an elitist but I’ve been around this business for a long time. There are many different kinds of developers, but I can assure you, only a handful will reach architect status and only a few those actually know design patterns. You can’t expect to hire all architects and senior developers. It’s just never going to work.

“We (meaning the smart people) have researched all of the requirements thoroughly. We understand all of our strengths and weaknesses. We know what’s best.” – By your divine right, you shall (cough) succeed. In the short-term maybe….let’s see your framework and code base in two years.

In summary, developers are the meat of any company that relies on computer systems for its products and services. Instead of putting one person in charge of the function of your business, doesn’t it make more sense to have everyone involved? Doesn’t it make more sense to rely on simplicity? Doesn’t it make more sense to create a technical environment that can be handed over to twenty Joe Average Developers and sustain productivity?

Or maybe you like your Ivory Tower.

Why Packaged ORM Frameworks are Bad for your Business

There are as many ORM or middleware or data access layer frameworks as grains of sand on the beach in Miami. Most or all do a lot of work for you and can speed up development tremendously. They can reduce code, increase quality, and centralize solutions that solve known problems in software development.

Most software architects love frameworks. For every architect, there’s at least one home grown framework. For every architect there may also be one or more favorite packaged framework. Some make a living pushing their framework by using it as a ramp up to consulting projects. Some write books about all the great patterns they’ve implemented in their framework.

Managers and directors and chief architects will all do powerpoint presentations showing how the chosen framework will reduce costs, save time, and improve results.

What they won’t tell you is that frameworks can be a terrible thing for your business.

Now that I’ve thrown down the gauntlet, let me explain.

I’m going to talk about Technical Business Debt today. This is from my own observations of using various frameworks in the last decade and I believe it’s something very measurable and very important to understand.

You have an existing codebase in any established business. At some point the code, the database, the reports, the systems…they all seem old. Somewhere along the line someone decides changes need to be made to improve one or more aspects of the codebase. You need a new data model. You need a web based data entry system. You need web services to share logic with other departments or external businesses. Whatever the catalyst is, something needs to change.

The first thing most architects look for is an easy win. How can I take the existing codebase and replace it or fix it with a cool new framework. Let’s implement an ORM system or Dependency Injection or Code Generation. I can have that new system developed in no time. Trust me.

Sound too good to be true? It is, but you won’t realize it until it’s too late. The problem with these pre-baked easy to implement frameworks is that you lose an essential piece of the upgrade process. You lose your business. How’s that?

It’s simple. If you slap a coat of paint on an old house, fix the trim, get a landscape crew to spruce up the yard, have some aromatherapy going on inside the house when people visit…hey, it looks like a nice house. People will be impressed. But they won’t see the cracked foundation that draws a 4 inch flood every spring. They won’t see the 15 year old furnace that costs 60% more than a new one would. They don’t see dust in the carpet that’s making all occupants suffer terribly from allergies.

If you really wanted to fix the house, you need to live in it. Then you need to take the time to fix things properly. You need to note the cost of heat and replace the furnace. You need to replace the old carpet with new carpet or hardwood floors. You need to put new windows in.

The same holds true of your business computer systems. In order to truly upgrade, you need to do it step by step. You need to implement each element of your business systems again, from scratch. The reason you need to do this is because your business has changed. Your customers have changed. The world has changed. As you rebuild your systems from scratch, the developers will ask questions. These will get passed on to a business analyst or stakeholder. The stakeholder will listen to the question and the proposed or assumed solution and stop everyone in their tracks. They’ll do this because the question is bringing up old business practices as well as revealing new practices. This process actually shows the stakeholders how their business has changed, much of which would have never been examined if it weren’t for the process of rebuilding each element.

It’s a bit like the Socratic Method. You learn by questioning and testing the foundation of each question. If you look at your systems and question their purpose as you rebuild them, you will discover how your business used to work, but more importantly, you will determine how your business should work. If you’re smart and lucky, you’ll also see trends.

Using frameworks that simply put a new face on old systems is fine for short term solutions. But don’t kid yourself that you’ve moved your business forward. You haven’t. You’ve just increased your Technical Business Debt. And you’ve also lost a priceless opportunity to examine your business at a low level. Something all businesses should do on a regular basis. You may not think you need to rebuild from scratch, but trust me, if your business has changed, the only way you’ll know is by doing just that.

Has Facebook gone too far?

I started using Facebook about a year ago. At first I was a little concerned about posting personal status updates, but adapted to it easily. I soon discovered that when I was feeling low, I would post personal information that I wouldn’t generally share with people other than close friends and family. I was a little nervous at first, but inherently, the need to commune overwhelmed the need to filter.

I was somewhat appeased by the ability to control who my “friends” were and that I could turn off any public dissemination of this information. That was a really big deal to me. I’ve made some comments on Facebook that I’d really not want left in Google Cache.

In the past year it has become clear that Mark Zuckerberg’s intentions with Facebook are to slowly erode the ability to keep personal information private. He’s stated on numerous occasions that he thinks privacy is overrated and that eventually there will be no privacy.

See here’s the thing. I was 25 years old once too. I think I remember where he’s coming from. He’s idealistic. He certainly has no concern for personal needs anymore. He can walk barefoot in blue jeans and a ripped up t-shirt for the rest of his life, like an American made Ghandi. But I think he’s looking at the world through glasses that no one has ever owned before and I’m concerned that way too many people are blindly following him.

I have a sneaking suspicion that after he gets married, has kids, sends his kids to school with other kids, he’s going to realize that there’s a point to this privacy stuff. He’s going to get caught in the conundrum, “What’s good for the goose is good for the gander.” only he’s going to begin deciding, with the power of wealth, that he would prefer to have parts of his life kept private.

When he turns forty, he’s going to look back at this moment and thing to himself, “Man, I was such a douchebag.”

The most recent changes to Facebook, called “open graph”, was the last straw for me. I was uncomfortable to begin with, but this is a clear assault on my ability to control my words. It proves that if you like your privacy, you’d better realize quickly that on the Internet, you have none. If you want to rant, save it for the family dinner on Thanksgiving. You’ll get it out, you’ll feel better, your family will laugh at you, and it will be over and done with. It will also be forgotten. Or maybe you go out with your friends every Friday night and let off some steam. It’s healthy. It’s not recorded.

I deactivated my Facebook accounts this past week without any warning to my friends lists. There have been reports of other, wildly more notable people, also deactivating their accounts. I assume the growth of Facebook will make deactivations a whispered cry.

But I don’t care.

My mind is my own. My thoughts are my own. My rants are my own. If I want to share something, I will call a friend or my mom or my kids. If I need to express myself, I have hobbies. If I want to like someone, I will write them a note or just tell them, “Hey! I Like You!”

You should think about this. No pressure. You’re connected. But you’re really not.

Is Flash Dead?

It’s an interesting topic these days. With Apple and Microsoft both saying that Flash on their mobile platforms is unlikely, where does that leave Adobe’s foundational development platform?

First of all, Adobe should and probably will focus all of its energy towards RIM (which has an alliance with Adobe) and Android (which just passed Apple OS as the number 2 mobile operating system). I’m sure Adobe is highly frustrated with Apple’s stance, especially since one could argue that a significant appeal of Apple computers has been as an artistic platform. Adobe has been the foundational component of that appeal with its design products Photoshop, Illustrator, and InDesign. If I were Adobe, I’d pull those products from the Apple desktop and market heavily on Windows. That might be just enough headache to bring Steve Jobs back to the negotiating table.

Microsoft is a different animal. Their mobile business is in reboot mode with the new Windows Phone 7 and it remains to be seen how successful the new platform will be with developers and consumers. Since Microsoft’s Silverlight is the development platform for the new mobile OS, it isn’t a stretch to see why they wouldn’t want Flash in their breadbasket.

But all of this is meaningless. Adobe Flash is still the standard on every desktop in the world and although their are a lot of mobile devices, people will continue to use desktops at work and in schools. The new fad is the tablet computing devices, some of which will run mobile operating systems, but it’s my assumption that within a year, there will be many of these running Linux and Windows, both of which support Flash. There will be iPads and other oddball tablets, but these devices will not replace netbooks, notebooks, laptops, and desktops.

I think Steve Jobs is overreaching and it’s going to cost him a lot of money. Apple is one hot competitor product away from losing marketshare and Windows Phone 7, WebOS and HP, and Droid all have an easy shot at being that product.

People think Apple is cool, but people don’t like to be locked out of choices. Apple is ironically playing the bad guy in their seminal 1984 marketing video. Someone will throw a hammer through their new playbook…it’s only a matter of time.

And Flash will survive just fine.

Why Rich Internet Applications are better than AJAX

I recently spent some time in a grindhouse development department learning Flex as well as supporting some basic AJAX features. I’ve also been working with Silverlight for over a year as a part of my Textfyre work. In the past, I’ve worked on projects that used some DHTML. I’m familiar with RESTful web services, jQuery, HttpWebRequest, and all of the other aspects of building highly interactive web apps. I’ve never done any real Java work, but I have played around with the technology enough to know that you can do decent things with Java (although the visuals have always left something to be desired).

Throughout these efforts I’ve always kept an eye what businesses might consider the most cost-effective solution for scaling their applications, whether the application is for internal use, external use, or a hybrid. Google took the lead in developing applications with AJAX when they introduced the nicer features of gmail.com and Google Maps. This was the birth of Web 2.0 and I would say we’re still in the middle of that era. A lot of companies have adapted to AJAX using jQuery and other toolkits. In many cases, using jQuery to liven up a website is very successful.

Then there are websites that are much more than just a website. We call these web applications and they tend to try to do many or all of the things that a desktop or “fat client” would do. These applications will load large amounts of related data, like customers, orders, transactions, and more.

A few years ago, our PC’s weren’t capable of handling this kind of scenario. It wasn’t the browser, but the lack of memory, disk space, and bandwidth. Today, most people have access to high speed Internet access. Most people have newer computers with more memory and more disk space. Certainly many businesses have either adopted these kinds of computers or will within the next year or two (in their move from XP to Windows 7).

So today we have the platform and the ability to make highly interactive and data intensive web applications. But the first question anyone should ask is, what are the requirements? I’ve seen a lot of applications ported from perfectly sound desktop applications (with deployment headaches, but still, they worked) to web applications. Deployment issues aside, there was no other requirement for the application to be built in web technologies. In many cases, the users lost many features and flexibility when their apps were webified. The process of moving a complex desktop application to the web paradigm is very expensive. We now have people called “information architects” that take very sound desktop user experiences and port them to web technologies. Why do we need these people. Because building smart web applications is vastly more complicated than building smart desktop applications.

There certainly are benefits to having an application webified. In a desktop application you might have a very strict set of menus and options or a single path to each feature. In a web application, you can develop multiple paths to features with varying usage scenarios. Some of this is possible in a desktop application too, but a web app offers these types of options through links and is much more seamless.

Benefits aside, web applications are very complex applications to build and moreso, to maintain. To allieviate the maintenance costs of building web applications, tools have been developed that make creating and debugging most applications very easy. Inline debuggers allow us to literally step through every line of code, see the stack trace, review element values, and resolve issues without throwing darts blindfolded. Debugging used to be a much higher level skillset. It’s still an art and skill, but with the tools today, it’s much more widely available to the average programmer. With an exception. AJAX.

AJAX is great, as I said before, for making websites behave nicely. But when you take the technology of AJAX and marry it to a complex web application, you’re exponentially adding complexity and cost to your application development and maintenance budget. The tools for debugging AJAX are not simple. In many cases a developer will resort to using Fiddler (a tool that watches HTML transactions) and arcane alert(‘here I am!’); statements haphazardly placed within the application’s javascript. This is a nightmare scenario for any developer. The more AJAX you implement in a web application, the more complex the interactions get and debugging capabilities rapidly deteriorate. And this is if your codebase is developed well and is highly readable. If the code was slapped together in a rush, forget it. You may never completely understand what the code is trying to do and it could be safer to do an organ transplant instead of minimally invasive surgery.

With any problem, someone inevitably sees it and develops a solution. Oddly enough, Sun developed a solution before the problem existed with Java technology. The problem is that Java made more of a name for itself in the server world and standard web application development. The client side features of Java were left to cute games and scientific demonstrations. You can blame a little of this on Sun for so desperately trying to make Java pure across platforms when they should have recognized the value of implementing a VM->Native compiler for each platform (Sun, Mac, Windows, Unix, etc). Instead, Microsoft saw the potential and created .NET, not necessarily to be cross-platform, but to enable managed code, garbage collection, for client-side and server-side development.

Eventually Sun realized the potential for Rich Internet Applications and has been actively marketing related solutions. But so has Adobe with Flex and Microsoft with Silverlight. These three platforms enable cross-platform application development that does not rely on web technolgies like html, css, and AJAX. These technologies offer front to back development environments with very robust and clear layout engines (arguing that CSS is hardly clear although it is robust), strong debugging tools, and deep integration with web services.

This is where application development is headed and it’s primarily because CIO’s have started to quantify the cost of developing web applications. We made enormous strides in resolving deployment problems (10 years ago we had to package software to be installed automatically on thousands of PC’s upon login and if there was a problem or virus, well, it was very painful), but at the cost of lost features and very expensive maintenance.

We now have the ability to deploy applications that more secure ane vastly more stable than before. Managed code, which is used in all three platforms, is the key to the stability issue. Security has become a primary focus within Microsoft and no one would argue that their systems and applications have improved dramatically since the early XP days of major viruses appearing daily.

We can now develop middleware that is adaptable and maintainable at a low cost. We now have the ability to develop disconnected presentation layers that can manage large datasets, but is also easily developed and maintained. It looks pretty too, if you have good designers.

If you look at new development requirements coming out of the major job sites like Monster.com, CareerBuilder.com, Dice.com, you’ll see that Silverlight and Flex have taken off. I’m not sure why Java hasn’t, but I assume some companies will adopt the Java desktop model too.

Throw away your CSS and AJAX libraries. You’ll save time, money, and your users and your infrastructure staff will love you. So will your shareholders.