Current Position and Responsibilities When I finished last semester and began this one I had recently taken on the role of Technical Lead for one of my company's development teams. The team is called Navy Training Information Management System (NTIMS).We build fleet training software for the Department of Defense (DoD). The project continues to be web based with web services technology and a mixture of client and server technologies.
I have continued to serve as technical lead for my team. It has been very difficult at times but also very rewarding. There are times when I go home at the end of the day and wonder where the last 8 or 10 hours went. The days are packed with designing, developing and planning current and future software to meet out client's needs. In my position I am responsible for writing code only about 50% of the time. That is tough for me. I like writing code and solving problems. The other 50% of the time is intended to contribute to planning, delegation of tasking, estimating time lines and a handful of other tasks. Honestly, I still jump in a write as much code as I can.
Progression I think I am always continuing to grow as a developer. I am always having to face new challenges and learn how to use new APIs to solve problems. Last semester we had recently converted our development environment over to use Microsoft's NET 2.0 framework. That has gone very well for us. While we haven't been able to take advantage of every new feature, we certainly have begun to leverage many of them. For example, we made heavy use of their built in AJAX technology. We also use generics which are very much like the STL in C++.
Aside from writing software and planning to writing software we are continually trying to make sure what have in place can grow as necessary to support a high level of network traffic and hundreds of simultaneous requests to the database. So far we have been pretty fortunate. We have made a few mistakes and had to revisit some decisions we have made that have caused significant performance problems.
One thing I can say is that not only have I learned new ways to solve common problems, I have also become more proficient at my problem solving techniques. I have come to realize that time is very valuable and sometimes the most well thought through, highly architected solution is not the best answer to a problem. In many cases you have to compromise between what provides the best long term solution and what will be the most efficient solution today.
Academic Relevance I would like to believe that the kinds of problems I talk about here are similar to those that other developers face in the careers. It is my goal to share my experiences and opinions with other cs students and programmers alike. Not to influence their decisions, but merely to offer a different perspective and potentially some enlightenment into our shared world of code related challenges.
A particular challenge I have had to face recently that was quite difficult to identify the root of the problem is related to a memory leak. We are always taught - almost from day one - that memory leaks are like the plague (you want to do everything you can to avoid them). We were using a component that was developed by someone else. This component was responsible for making asynchronous request to the server from client-side code. This is technology I speak on a lot. It is Asynchronous Java-script And XML (AJAX). We were using this component - as mentioned last semester - to poll the server every 5 seconds. We had worked out all the performance issues with the polling implementation, except one. The memory leak.
For the life of me I could not understand why my code had a memory leak. I was deleting all variable references when not needed. I was setting disconnected object to null when no longer attached to HTML elements to avoid closures. I could not solve it. I lost sleep over it. I finally threw in the towel and decided to stop the polling after a few attempts to minimize the damage of the memory leak. the leak was in the amount of 8k per request on a 5 second interval. That means that every five seconds my code was using 8 more kilobytes of your computers RAM. Some people stay logged into the system for hours doing work. So lets look at the magnification of the problem:
Logged in time: 2 Hours = 7200 seconds Request interval: every 5 seconds Request leak size: 8k per request ---------------------------------------- Total # of requests (TR) = 7200/5 = 1440 Total memory leak (TML) = (1440 * 8kB) = 11520KB or approx 11.52MB
Finally, I had a thought. I know java-script very well and underneath this component it was using java-script to generate the requests. So, I disassembled this component and started poking around. After about 30 minutes of searching, I found the problem. A new request object was being created and added to an array each time. It was not correctly being de-referenced and deleted from memory. The dereference code was there but not correct. So, I fixed the logic, recompiled the assembly and tested the memory leak. I let the code run over night. When I came back in the morning the all memory had been correctly reallocated and the system was running efficiently.
That experience was once of my most rewarding to date. After stepping away from a problem for a while and coming back with a new perspective, I was able to look at the problem in a different way.
Summary I have enjoyed doing this coop for several reasons. It has allowed me to share my own experiences with anyone who dares to read my blog. It has forced me to think about my work in way that I can communicate it to others. It allowed me use my writing as a way to reflect on my growth as a student and a developer to see where I need improvement. I highly recommend this experience to anyone willing to take is seriously.
Light at the end of the tunnel
Another semester is quickly coming to an end. For me, I am usually less worried about time going by too slowly and more worried about trying to get everything done in time. I am not a procrastinator. I attack obstacles head on and start as soon as possible.
The biggest problem for me is that I take on a lot. I always find myself with a ton of different things to do and only a limited amount of time to do them in. I am sure many of you can relate. If any of you work full-time and are classes part-time or vice-versa, then you know wheat I mean. It is hard to be a good husband, father, student, employee and any other role you may have.
I am just saying all this to complain or get your sympathy? No. I am just saying hang in there. I know your pain. I feel it too. I will be happy when I finish my degree and spend more time with my family and become an expert in my field. I think it is all worth it. It is just hard to see the light at the end of the tunnel sometimes.
The Right Stuff
Recently, I had the opportunity to conduct a handful of interviews for the Company I work for. These particular interviews were for software engineering positions. We were looking for candidates with varying skill levels to fill different levels of positions (Junior, Mid, Senior, etc.) Needless to say, I was a little disappointed in some of the results.
I am a pretty easy going guy who likes to give everybody a fair chance. However, there comes a point where you have to ask yourself the question, "If I recommend hiring this person, how long will it take before s/he can be productive?" On more than one occasion I found myself coming to the conclusion that some of the candidates really needed to go through software engineering boot camp. They lacked some of the fundamental concepts important to Object Oriented Programming (OOP).
The reason I am disappointed is has nothing to do with me thinking I am better than anyone else. Or, thinking I have paid my dues and everyone else has to as well. It is really because I am not sure I believe CS majors enter the field with the right skill-set to become active members of the software engineering workforce. I have begun to think that there are plenty more skills a developer needs to be marketable in today's job market.
That being said, I do realize that CS curriculum is intended to be pure computer science. It is geared towards teaching you the theory behind computers, enough math to solve a variety of problems and understand complex algorithms. It is there to be a springboard not into a particular programming language, but the fundamental basis to learn any language. I get all that. What I don't get is how a guy fresh out of college comes and applies for a job with no real world experience in a programming language he has never seen and expects to hit the ground running.
I have honestly had better luck with candidates that are self taught - many times having degrees in other fields - that have real world experience in a variety of languages. Many times the real CS graduates can solve certain problems theoretically but have a really hard time translating that into actual code-able solutions. I am not trying to sound like some almighty programmer that is too good for everyone else. In fact, I make just as many mistakes as everyone else. All I am saying is that I think CS majors are mislead, in that they go through 4 years of school thinking that they can walk right into a job in some language they have never seen developing for an environment they have never developed for before.
I think they can certainly learn, but the curve is steep. If I had gone to school first and gained experience later, I feel like I would be lost right now. As much as I would have like to finish school many years ago, I am thankful to have been in the field and have all the classes I am taking be more meaningful to me.
If p, then q
I am currently enrolled in CS381(Discrete Structures). It is always interesting how learning new ways of looking at the same information helps you in your everyday work. When I first starting reading my discrete math book, I made several assumptions that proved to be incorrect.
First of all, I made the assumption that the conditional proposition p -> q (pronouced, "If p, then q") was like an if block in your average programming language. They are similar but not the same. Or maybe they are the same but different. Or... Anyhow, in your everyday programming language the statement if(CONDITION_TO_BE_EVALUATE_EQUALS_TRUE), then DO_THE_FOLLOWING. Some langauge literally use the words if/then and others just use if and { } syntax instead. For this discussion the syntax is irrelavanet since all programming languages try their corresponding version of the if/then the same.
Furthermore, the preceeding example if an if/then can probably be better illustrated with an actual example. Let's use an actual proposition to make our point. We will use the following two propositions: Proposition P = I stub my toe Proposition Q = I am bleeding Now, lets use these statements in a condition. To use these staments in a C based programming langauge it would look something like this: ------------------------- if(I stub my toe) { I am bleeding } ------------------------
In other words, "If I stub my toe, then I am bleeding." P -> Q. Seems like they both match. The discrete math version seems to be indentical to the programming verison. Well, here is where I went wrong. If you write out the truth table for P -> Q, you will find that only time the truth value is False is when P = True and Q = False. That means that no matter what, Q still gets evaluated. But in my code block above Q only gets evaluated if P is True.
Here is the truth table for P -> Q:
P | Q | P -> Q T | T | T T | F | F F | T | T F | F | T
So, the assumption I was making is that I would only be bleeding if I stubbed my toe. According to the scope of my programming if/then block that would be the case. However, you could still be bleeding from something else. Which is also true in programming. It is very possible for some other expression to evaluate the same data. I just did not open my mind enough to realize the discrete math side of the logic.
I still have a long way to go to having a thorough understanding, but this was certainly one less thing I had to worry about.
There is more to being a good programmer than writing code
I have met a variety of programmers in my experience. I have met amazingly talented programmers with little or no academic programming experience. I have met programmers with BSCS and MSCS degrees who have very little real world experience or the ability to solve real problems. I have also met programmers with levels of experience and talent anywhere in between the two extremes. The first point I want to make is that going to school for a Bachelor's or Master's degree, obviously does not guarantee success in that field. And secondly, that programming is a field that does not require academic experience to be extremely successful. It takes more that writing good code to be a good programmer.
That being said, I have my own thoughts on being good a programmer and what it really takes to make a difference. Before finishing my ASCS at TCC and starting back ODU to work on my BSCS, I had already been working in the field for a few years. I was primarily self taught with a handful of mentors and problem solving skills. I worked with some guys who had never been to college and could solve programming problems that would rival MIT graduates. I believed there was hope for me too. But after a few years I began to struggle with deeper concepts. I needed to know why things were happening a certain way. I wanted to be able to make reasonable assumptions about why one technique would be more efficient than another. This led me to go back to school. Not to mention, I wanted a piece of paper in my hands that stated I paid my dues and earned my place in the computer science field.
Now that I am most of the way through my BSCS and still chugging away a couple of classes per semester, I look at problems differently. I tend to take a more scientific approach to solving problems. Less ignorant guessing and more educated hypothesizing. Either way, it might be a guess but at least the latter stands a better chance of success. Chances are if you are trying to solve some problem with code, you are not the first one and there may be a pattern out there that matches your problem very closely (these are called design patterns). Knowing where to look and what your are looking for are some of the most important things any programmer needs to know.
Which brings me to my next point. Just because you can write code to solve a problem doesn't mean you have solved it correctly, efficiently or in a way that other programmers coming behind you will understand. It is very important to lay a solid foundation any time you put your fingers to the keyboard and begin writing code. I see guys all the time that go out on the web and copy someone elses code to solve a particular problem without knowing how it works. First of all , this is a bad idea. And second, this is a bad idea. Did I mention, that it is a bad idea? Anyway, if you are stuck and can't solve a problem, by all means go lok for code that does what you need. But take it a step further. Try to understand why it works, how it works and that the outcome of it's use is predictable.
You will be a far better programmer for using this technique. It will help you learn and grow your skillset. Who knows, next time you solve the same problem you may not even have to look for examples. If you figured it out the first time you might be able to work through it on your own. Slow down. Show your work (as our professors always say). You will appreciate it later. I promise.
Begin with the end in mind
That quote "Begin with the end in mind" has always been one of my favorites from Stephen Covey (Motivational Speaker). It relates to the bigger picture of things and less about the minutiae. It is not just a phrase, but also a mindset. It eludes to understanding what direction you are going before embarking on your journey. Now, one could also argue that life is a journey and not a destination. That may also be true. But when you put it in software development terms, you very rarely want to set forth without knowing where you are headed. That can get expensive and cause a lot of rework.
All that being said, I apply this approach to my daily work. I often try and understand how my work will be used or the future needs of the code I write before I go off and start blasting out lines of code. I believe it is a very sound approach to consider the lifetime of a given unit of code before investing too much time into writing it. If you are writing a simple little utility to save you having to write the same code over and over, it might not require much thought. However, if you are writing a small piece of a very large system you need to think about all the entry and exit points to your code.
Obviously, there is such a thing as over-architecting a solution for functionality or features that may be overkill or never get used. This tends to heppen a lot in the OOP world. Developers often get in the habit of trying to make their bullet proof code comepletely scalable, resuable and fully modular. This a great thought, but usually difficult to implement rarely provides worthy return on investment (ROI).
Just stop and think before you go fingers to keyboard. Ask yourself, "How will this code be used?" "Are their techniques I can use that will make my code resuable and make the development process more efficient?" Over time you will develop a set of common practices that will feel like second nature. It will make the process easier and more predictable.
Calling all unit tests
Writing good code is something that you should become more and more proficient with over time. In order to be proficient it takes practice, attention to detail and a willingness to wade through hundreds of lines of code to find problems. Debugging your code is essential if you have any desire to ensure it's worthiness. If you are gonna write code and not test it thoroughly, then you might as well send out a disclaimer stating something to the extent of "I wrote this code and it might work correctly, but I have no idea because I didn't really test it." At least that way anyone using your code or interfacing with will have an idea of the can of worms they are opening.
That is not to say that every line of code you write has to account every unknown scenario it could be subjected to. That would not only be impossible, but also huge waste of time for which you would likely never see a return on your investment. Code has to evolve over time. The system for which a particular code base was written may have new needs that exceed the capabilities of the current code. That will cause the code to break, and it may never have been able to be anticipated.
What this does mean is that you need think about how your code will be used. A basic place to start is to identify inputs/outputs and precondition/postcondition scenarios. Obviously, you cannot predict every single scenario in which your code will be used. Once you have identified the main uses of your code you can start thinking about how to test it. There are several ways to do this.
Unit tests are a great way to make sure your code is functioning according to the interface you have provided. For example, I write a lot of code in Microsoft.NET's C# language. I use a freely available piece of software called N-Unit for unit testing my code. N-Unit is a unit testing framework for the entire .NET family of languages that lets you write any number of tests to ensure the validity of the code you write. There are several other unit test frameworks available for other languages.
Without attempting to give a lesson on creating unit tests, I will give a brief overview of the idea behind creating tests. Most unit test frameworks have a way to identify setup, tear down and individual tests. For example, in N-Unit you would decorate methods in your class with attributes like [SetUp], [TearDown] and [Test]. Then when you launch the N-Unit application you choose the assembly (compilation unit) that contains your tests. N-Unit will find all of the methods with decorations and allow you to choose them for testing.
So what should a test contain? To be of any use, your test should contain logic that can validate the purpose of the code it is testing. Think of it like a mathematical proof. It is a way of providing a general set of assumptions that can summarize a solution to a problem. If you can write a test that correctly validates the range on inputs and produces a consistent output, you have a proof; your unit test is successful.
In summary, it is important to try and ensure the integrity of the code you write. Unit tests provide a way to do that. Over time you can maintain a repository of your tests and run them any time your code base changes. This will alert you to any oversights or flaws in your logic. So get to it; go write some unit tests.
Picking up the pieces
I have been very busy at work. After transitioning into a new role, the days seem to go by even faster than before. I have chosen to continue this blog for the Fall 2007 semester. I hope to share some of my work experiences with the rest of you - whoever you may be - out there. Working full-time, going to school part-time, being a good husband and a decent father to my two boys is certainly a challenge these days.
Life is constantly like a juggling act; a delicate balance between zen and complete chaos. I have to stay very organized to even think about getting everything that is needed to be done in a single day. I have about 12 classes left until I finish at ODU. While that would seem exciting for most people, it tends to look like a long journey for me. At a rate of just a couple of classes per semester , you can see how it my drag out.
I find Computer Science to be simply amazing. I am so intrigued and inspired on a daily basis to accomplish new things or make existing software better. I care a lot about the quality of work that I produce. I always try not to settle for the first answer to a problem. Instead, I try and find the best solution, within reason. Finding the best solution to a problem takes a lot of time, effort and often leads to a lot of failure before you achieve success. But in the end it is worth it. You own the right to say you did it. You solved a particular problem in the best way you could.
Anyway, I just wanted to get the semester started with a brief blog entry of what's going on and share some of my current thoughts. I look forward to sharing my programming experiences with you over the next few months. I may even bore you with some of my personal life and irritate you with my idealistic opinions.
In the event that you arrived here by chance and have no idea who I am or what I am talking about, you can go back in time and look at my first entry here.
Final Report
Current Position and Responsibilities When I started this Cooperative course for the summer semester I was a Software Engineer on one of my company's development teams. My responsibilities included designing, developing, testing and deploying web based software. The company standard server side language is C#. It is one of the Microsoft .NET languages. Additionally, we write client side-code using javascript as necessary for client-side interaction.
Since then, My role has changed. I am still a developer on the same team. However, I am now the technical lead for that team. It is a huge responsibility and a great opportunity. I aspire to live up to the expectations placed upon me. My new role includes many of the same responsibilities as before, as well as many more. Now, I am also responsible for scoping out what functionality will go into what releases of our product, assigning tasking, researching new technology for use in our software and coordinating our efforts with the project manager to meet customer expectations.
Progression Over the past several months I have continued to grow as a developer. We have recently undertaken the task of converting our entire web based application to a newer technology. This required a lot of coordination and planning to migrate our code base and keep our application stable. We previously used Microsoft's .NET 1.1 framework. Our goal was to convert it over to use 2.0 version of the framework. This afford us language benefits like Generics, Partial classes and new control library. While most of the conversion went smooth, there were some deprecated areas of code in our old version the required us to figure out the new way of doing things.
All of this new technology has given me the chance to learn different ways to solve problems and enhance my skill-set as a developer. There are so many ways to solve problems and the more experience you have, the better the chance that you will find a good way to solve a particular problem. One of the biggest benefits is that I have been writing a lot of code. The more code you write the more proficient you become and the more familiar you become with common patterns of development.
Speaking of patterns, I have been spending a lot of time learning about design patterns. Design patterns are basically known solutions to common problems. The idea is that many of the problems we face everyday as programmers can be broken down into smaller identifiable pieces that are more easily solved. These patterns have names like Observer, Singleton and Factory. They have categories depending on if they are creational, behavioral or structural.
Academic Relevance It would be difficult for me to pick a single problem I have faced to illustrate the relevance if this blog, this coop, or even my employment as a software engineer. Throughout this semester I have tried to touch on subject matter that I think pertains to the kind of challenges many programmers will encounter. If I had to pick a particular problem I have solved there is one that comes to mind that I have worked on recently. This problems is related to how we keep up to date real-time information on pages throughout our application. About a year ago I wrote a messaging component that polled a database for changes related to processes that were going on. They component was very successful with a minor problem; the polling. Here is the magnitude of the problem: For every logged in user a thread would poll the database every 5 seconds checking a rather large result set for changes that pertain to that user. Let's say you have 100 logged in users. This means that every 5 seconds there are five hundred request made to the database. You can see the potential performance problem.
After some analysis and collaboration with team members I came up with several separate items to help solve the problem. Once piece of the solution was to store the results in a cache in the application and only have the application as a whole query the database every five seconds. That way the messaging component will just poll the cache every 5 seconds. This offered great results almost immediately.
But there was one more thing that could be done to help this solution be even better. By adding another table to the database that just keeps track of the change timestamps of the data in question, we could reduce the queries to the large data itself. To make this happen we used a trigger in the sql server database. A trigger is like an event handler; it allows you to monitor database tables/columns and invoke commands when something you are interested in changes. We leveraged this to update a very small table that just keeps a time-stamp of the last time the data we need changed. So, every time the application polls the database (every 5 seconds) if the time-stamp is not newer than the last it checked it doesn't try and fetch any new data.
There are still a few more things we could do make this solution even better. But these two things gave us about a 70% performance increase from the test we ran. I would call that a success.
Summary I intentionally chose to do this Coop to use it as an opportunity to share my experiences on the job with the rest of the CS students out there that may have an interest. We may all write code in different programming languages but at the end of the day, the constructs we use are primarily the same. I gain so much useful information from reading blogs of other developers that solved the problems I encounter everyday. My goal was to provide some insight into the world I work in everyday and inspire early or mid-level CS students to hang in there and keep solving those problems. Computer Science is a challenging field that does not always lend itself to instant gratification. It requires a lot of patience and creative problem solving skills. This semester has helped to remind me of that. Thank you for reading.
What does the future hold?
Work has been really busy lately. In fact, it has been hard to keep up with all of my regular work and my school work. My Company recently did some orgazational realignments. We have some new projects starting up and they need staff. What does that mean to me? Several things, really.
First of all, three .NET developers and one Database administrator were moved to another team. One of those developers happened to be the technical lead for our team. OUCH! That is like taking away my legs. He is a good friend of mine, a great mentor and an excellent developer. There was an opportunity for him to lead a new team get this new project off the ground. If anyone can do it, it is him.
Next, we now needed a Technical Lead for our project. I was offered the position. I was scared to death. The position is a lot of responsibility and an overall huge commitment. After giving it some thought I accepted. I decided that it is a great leadership opportunity and a chance for me to really grow as a developer.
Of course, I have to ramp up on several projects that our team is now responsible for. Not to mention, attend more meetings and still write code to make sure we meet our deadlines. I am excited and nervous all at the same time. I guess if you don't take risks you don't get the rewards.