What I've learned this semester in 532

While hopefully I have learned a lot, from many different areas this semester in this broad topic of a class, one all-encompasing lesson I have learned is that Enterprise Systems are much larger than I had once anticipated, but they are within our reach of understanding. While the specific topics covered were sometimes scattered and seemingly unrelated they all did relate in that large corporations and organizations need complex solutions like we talked about to serve their own and their customer's needs.

One of the best culmination points in class was after we had learned about some of the technologies that large companies used to see the groups that presented on "The architecture of Wikipedia", or of Google where we saw these scaling principles in use by today's technological giants. Because my group was assigned the topic of scaling a PostgreSQL server I was forced to think outside my typical LAMP environment and see the benefits of other products that were built on a different foundation but for similar reasons as the products that I was already using. This research helped me to expand both my knowledge and the selection of services that I could offer to potential clients.

Thanks to Dr Liddle's PHD work and his continued involvement with MDA we did spend considerable time on this concept that I had never been exposed to before. The idea of generating complex platform specific applications from detailed platform independent models sounds like a lofty goal, but compared to other innovations in efficiency today it sounds like a natural step of progress. After learning from both class and independent research about the concepts and actual implementations of MDA the idea does not seem too "pie in the sky" and I have considered building such an application to aid my own professional development.

As I work towards graduation and start my job with Ernst & Young's Business Risk Services group specifically performing internal and external audits on large companies in the Seattle area I'm sure I'll see many of these technologies in use. Coming from the Seattle area and more specifically Bellevue and Redmond where Microsoft has their global headquarters I was indoctrinated with the MS way of doing things, thanks to classes like Enterprise Architecture and others I have developed more of an open mind about the possibilities of technologies and languages. Microsoft will always have a place in the IT infrastructure of corporate America, but their presence is evolving and even shrinking in many companies so a foundation on just Microsoft technologies will not get you as far today as it could ten years ago.

All in all the experience as well as the knowledge gained in this class will better prepare me for both my professional career with Ernst & Young as well as my freelance career as a custom web developer. With people changing careers an average of five or six times before they retire I'm sure I'll have the opportunity to experience, and maybe even develop/design many different information architectures in many different industries. The foundation that I've learned not only in ISYS 532 but in the entire MISM program has prepared me to take those challenges and overcome them successfully.

Amazon Web Services

The services that Amazon is offering to anyone and everyone are pretty incredible. To imagine with a few bucks I could fire up a group of servers on another continent to do processing that I specifically ask it to do when I want it to do the work. Unfortunately I think that Amazon is a bit ahead of its time and that the turkish robot and web services are beyond the scope of most large businesses, but it does give businesses of today the ability to plan around services like this being available in the future.

I could see myself using these services to run batch processes and calculations that are run on an infrequent schedule, that way I don't have to maintain the servers in house or maintain a contract with a provider for a service that I only use occasionally and temporarily.

As far as making money on the site I don't see myself doing little tasks for two to twenty cents apiece and feeling like I had really accomplished anything. These repetitive tasks that appear on the list should be figured out with game based processing or some other kind of way of receiving input as opposed to paying random stratified people for really just wasting their time.

Logging & Messaging

These class periods that covered logging and messaging were very helpful because I have been having a hard time figuring out how to better manage my server logs and handle errors in my applications. It seems the logging on my campus servers aren't working correctly and if I ever want to track down an error I have to enable the displaying of errors temporarily. Displaying errors is not a good thing to do on a production site, but when logging doesn't work it is the only way to figure out what is up.

The only time that I actually used messaging in error logging is when on an application for campus employees where there was the ability for users to guess a specific code and possibly change some nonpersonal reporting information of someone else. When a user alerted me of his malicious discovery I implemented a system that emailed me of the modifying user's information and the details of the attempted modification as well as an email to them telling them that they ought not change that information. This was a little interesting because I was actually catching people in the act and kind of accusing users of inappropriate use of the site which I had never before done.

Ajax

I think we talked about ajax, anyways I love ajax and think that all data intensive interactive websites should be using ajax by now. Anytime I can convince a client to use ajax, or I have the budget for it I do implement ajax and the clients are always satisfied. One example of ajax that I did recently was for a non-profit's auction registration form http://mh.realwebstrategy.com/mammashands.org/index.php?action=register

Of course there are some problems with ajax, most commonly when the user clicks the back arrow to reverse one step but instead finds out they have erased all progress made within the ajax application. This is a frustrating issue and has encouraged me to develop some standards of easy to use inner-application breadcrumbs, as well as saving during the application and breaking the app into non-ajax parts so that the back/forward buttons do work occasionaly. I have read many posts and articles about how to resolve these problems but none have been straight forward enough for someone of my somewhat limited javascript background to handle.

My newest craze in web development comes from the flash class that I took this semester. I was having a hard time grasping the concepts of all of the graphical items and movie clip controls until finally we started coding in actionscript and used classes etc. But the best thing I learned in that class is how to use flex, adobe's new framework for building data rich applications in flash using a markup language MXML similar to Microsoft's XAML. An example of one of my creations is here at http://www.lesliehancock.com/flash it shows a photo gallery and digital/printable flier of real estate properties.

I will continue to develop in flex which is really ajax by nature. This way I can deliver more value to my clients quicker and have the final product not only work flawlessly but look professional and trendy!

Web Services

This assignment was tons of fun, but tons of work. I got out to what I thought was a fast pace and hooked to Dr Liddle's service and returned the content back to another page in about an hour, the difficulty came when I attempted that process in wsdl mode. I helped a classmember out and ended up helping myself in the process. WSDL caching, different versions of PHP SOAP and other issues kept my service from working correctly till almost the deadline. Anyways I finished it up and actually enjoyed the process, and now that I'm moving a lot of my professional development to flash and more particularly flex and I still need to access my data through php web services will be a good technique to accomplish that task



In case I was supposed to blog my assignment instead of email as I did on 2/19 here are the contents of my assignment submission.

Dr. Liddle,

I finally finished my web services assignment. Here is a link to the wsdl http://dev.realwebstrategy.com/532/finalservice.php?wsdl

I also set up a thin client to do what your test client did and accept a cityzip get parameter http://dev.realwebstrategy.com/532/finalclient.php
getWeather($cityzip);
print "

";
print_r($return);
print "

";
?>

Thanks,
Michael Hancock

MDA Part 3

The specific product that I think is most interesting in the MDA arena is Microsoft's software factories. While the Microsoft product is not specifically and MDA tool and actually Microsoft has publicly said that UML and MDA as defined by the Object Management Group are too general their software factories that are based on DSLs or Domain specific languages are very much a portion of the MDA framework. OMG is very ambitious to think that every problem can be produced on any platform with a handful of different modeling languages, that is why Microsoft, one of the if not the largest and most resourceful software development company in the world is backing an initiative that it feels is realistic and will be profitable to us developers. Below are some images describing how Microsoft Plans to implement these software factories using domain specific languages.




One thing to note about microsoft right now is that they have not released any software factories, so even though they are not as ambitious as OMG in their MDA aspirations they are still working on a working implementation of their specific tools. Because of this there is no pricing information available, or even a good idea of when these software factories will be available.

MDA Part 2

I currently run a very small web development business and pick up jobs when I have time and clients, unfortunately I tend to get all my clients at once and end up refusing work that I needed just a few months earlier. I think Model Driven Architecture might be a way that I could work faster, in a more documented fashion, and involve other developers in my work easier. Lately when I have tried to include other developers I have to give them a totally independent piece or I tend to rewrite their code, I would rather be able to give them the specifications and have them create a model that would generate the architecture in the manner that I have configured my MDA tools.

I would not attempt to implement MDA in small odd jobs that I don't anticipate adding to, or getting any additional help on. These sort of jobs I can hammer out in a week or so and usually do not have the lifetime requirements to justify a completely solid, documented foundation. Whenever I do involve multiple people or perceive that my work will be part of a much bigger project I would like the benefits of a well-documented platform independent model like MDA. This way I can easily scale a project, retool a project by modifying just the model and reexecuting the conversion tools to produce the finished project.

This may sound like I'm asking for more trouble than I would be saving but for my purposes I think I would create my own interpreter to take the model and produce the code in the language platform that I am most familiar with. This method would allow me to keep well maintained the product and debug and optimize the application if there are any problems or ever room for improvement. I have already created an code generation engine where I tell a system the specifics about my object and its relation to others and this produces a Business Object Class, a Data Object Class, an admin page to administer the objects in the database as well as a function to be used when this object is referenced as a foreign key. The url to this page is http://wellness.byu.edu/includes/codegen.php (there is a password to keep unauthorized people from writing content) . I have thought about building a tool that uses ajax to tie many objects together and then produce the backend code that is all interrelated and will do this if I continue to receive requests to build larger object-oriented systems.

MDA Part 1

MDA is trademark and sponsored by the Object Management Group and its consortium. OGM also trademarked many other model driven terms so that they could apply their strict standards to anyone adopting any part of their model. As far as I can understand Model Driven Architecture is code generation, (architecture generation) from a model built in one of the many available modeling languages. These models should be platform independent and by using MDA tools can be converted into any number of platform specific models which in turn with another mda tool will produce the actual application architecture defined in the model.

MDA is seen as a solution for many of the problems dealing with constantly changing development languages, platforms for applications to run on, and organizations tied to proprietary products because the learning curve is too high. Now a high level platform independent model is designed which is then translated into any number of platform specific models and then into the application, this enables the application designers to work free from the constraints and specifics of any particular language or platform.

In class we have learned plenty from the OlivaNova Model Execution suite, but there are many other vendors that produce tools, or packages of tools to be used in an MDA environment. Microsoft obviously cannot claim to be platform independent as they consistently refuse to develop on any platform than their own but they do provide tools called software factories that produce software based on a .NET DSL (domain specific language) produced from a platform independent model.

DB Opt

There are two ways to go about query optimization. The first way is to write more efficient queries by avoiding too many joins, not asking for columns or any other data that you don't need and keeping the number of tables down to a minimum. The second way to optimize a query is to help your database to be ready to handle specific often used queries in as fast a manner as possible. During this assignment I will explore both methods of query optimization and hopefully bring some performance gains to this database and the often used queries.

To be able to investigate in depth how to improve the queries used on this database I will select a limited number of queries, a few short and a few complex in order to produced a detailed guide to optimizing those queries. Before any optimization takes place I will benchmark each of my queries so I know exactly how long each query takes to execute and also record the initial states of each query before I make any changes in an attempt to improve query performance. I will also disable the query cache so I know my results are not skewed by MySQL's attempts to speed the query process.

Now that I have taken my few queries, benchmarked their current performance and their initial text I will look for issues that could be keeping those queries from performing like they could. One thing about this particular database is that the only indexes right now are on the primary keys. This is an obvious area for improvement and some of my fixes will involve adding a few indexes on short fields that are referenced in these queries. Simply adding an index onto every field will nullify the optimizing by choosing indexes because the index table will be just as large and slow as the unindexed tables.

One area that causes a lot of problems on poorly built MySQL queries has to do with subqueries. Subqueries actually perform multiple queries inside of a single query which steals processor power and memory space. The best alternative for subqueries is to use a standard join whenever possible, also the MySQL optimization engine can accurately tell what kind of join will best speed the process up so there is no reason to explicitly tell it to do a straight join, or any other kind of join. This database is fairly optimized in this regard, most complicated queries involve joins instead of subqueries but there is a little room for improvement that regard.

Focussing on schema design there are a couple things that should be avoided, bad index choices including multiple indexes take up room and sometimes do more bad than good. Also oftentimes we don't think about the column size when we make it an index, and when we make an index on a char(200) or bigint field we are requiring every entry in the index to be of that size. We should break apart large chunks of data into smaller more manageable, more searchable portions that will optimize our database performance.

The last way that we should optimize a database performance deals with actual server parameter changes and hardware upgrades. Most of these changes will only make a minimal impact on the performance of poorly written queries but these will help give you that extra edge on reducing query times and processor, memory usage.