Trond-Eirik is rambling on about stuff

SOA does not belong in a system architecture

2008-11-27T23:18:00.001-08:00

It seems as the words of the SOA hype is starting to fade off, and people is getting more realistic expectations to what SOA actually is and what it can do for them.

From my point of view there seems as if a good number of architects have done a number of pretty bad design choices involving SOA. Often SOA is understood as something that should be used within a single system as a mechanism to separate the different layers of the system.

SOA = Service Oriented Architecture.

That doesn’t sound too bad, does it? Orienting our architecture around services? How can that be bad?

Actually, it might be that bad. The problem might as well be how people understand the realization of a service. Many consider the following statement as true:

A Service in SOA is the same as a WebService.

The other definition that is a bit less extreme is to look at a service as a contract and layer boundary.

Anyway, the practical result of this is that what goes in or out of a service is data. By data I mean dumb data. It might be formed as a Xml document, as string, as classes or other structural information. The point is that it is data structures that do not contain any business logic. Actually, that’s one of the main purposes of SOA: To separate the logic from the data as well as separate the technology dependency from the data, enabling different system to communicate between them.

So, on one side we have a method that is able to transfer dumb data and that is suited to communicate between systems built on different technology and different times.
On the other side we have architects trying to use this as a communication pattern between different layer of systems using the same technology, written at the same time and running in the same environment.

Why is this bad? This is bad because at the second you introduce a service (especially if it is realized as a web service) in a system you end up with a flow of logic and data that goes like the following:
- Service read data from datastorage
- Service internally structures data in a good way, probably in a model
- Service internally verifies data structures
- Service converts data to a suited dumb external format
- Service passes data on to the client
- Client receives dumb data
- Client internally structures data in a good way, probably in a model
- Client internally verifies data structures
- Client changes some data
- Client verifies business logic (some of it)
- Client structures data in the format wanted by the Service
- Client sends data to the Service
- Service internally structures data in a good way, probably in a model
- Service internally verifies data structures
- Service internally verifies business rules and logic
- Service saves data to data storage

This seems bad. And it is! It’s actually awful. The added cost of this type of logic is extreme. The cost is both in pure code realization time, bug tracking, runtime speed and the agility and ability for the system to change.

However, in some cases this is actually needed. But that is when you want to have interaction between systems that is very de-coupled and not actually related. It might for example be that you are communicating with a system in a different company. That company must be able to change it’s internal code without affect other companies. It might also, in some cases, be a suited way of communicating between systems in a large company.
But it IS NOT for communication within a single system, not even if that system is big. If you want to archive separation in your system, use other patterns.

SOA is a good idea, but it has to be used at the right places, and IMO that is not within a system architecture.

Completed the certification, ready for the next round

2008-06-17T04:50:00.000-07:00

A while back, when I started working for Fundator, I started a certification run. The purpose of this was mainly to document my current knowledge and at the same time broaden the knowledge area a bit.

I have now completed this, by taking 6 MS exams, and I do currently have the following MS certifications:

Microsoft Certified Professional Developer
Enterprise Application Developer

Microsoft Certified Technology Specialist
.Net Framework 2.0: Web Applications
.Net Framework 2.0: Distributed Applications
.Net Framework 2.0: Windows Applications
SQL Server 2005

As you of course will notice, these certifications are already a bit outdated. The hot new stuff now is .Net 3.5, WPF and those certifications. I have also noticed that MS have just introduced their new Master series of certifications.

However, the area of knowledge needed for the new MS certifications is mostly extensions of these certifications, so I do absolutely think that these certifications will be good to have.

Now it will soon be summer holyday. For now I will stay low and relax, but after the summer I'll probably continue my certification run, but then it will be a lot more learning of new stuff...

Early prototyping

2008-06-16T13:28:00.001-07:00

It haven’t been a lot of coding for me lately.

I’m currently working on extending an existing product with a new area of functionality. This involves a lot of studying and learning new stuff.

Some of the most exiting parts of this process is to notice how you gain more knowledge and understanding about the problem area as you go ahead and learn new stuff. I have been through this process several time before, but this time is a bit different as we are using some more time on the learning process. We are working in a broader area and there is no expert or customer that has full knowledge of the problem area.

The task at hand is to carve the product into shape, figure out what it should do and how it should do it. Kind of creating a product vision.

As a part of this I have done some very early prototyping. Earlier, when working with more concrete solutions, I have found that using Visual Studio in combination with bitmaps have worked relatively well.

In this project that did not work at all. The outcome felt "to concrete", it did not leave enough room for the next phase and it quickly become too close to a real solution while at the same time being difficult to use.

Based on that experience and some more testing I’m currently using several tools to create GUI prototyping.
- I’m using paper/drawing to test out initial ideas and thoughts
- I’m using excel/word to test out special situations and presentation
- I’m using paint to steel small pictures/figures/symbols and ideas from other products
- I’m using power point to put together all elements, as well as adding simple animations to show transitions and illustrate the most important navigations

I’m sure that there is a lot of other GUI prototyping tools out there. However, to practically solve raising issue, I have found this method, using simple and available tools to work quite well as a way of sharing ideas and possible solutions with others.

What's happening with the portable devices

2008-02-12T00:16:00.000-08:00

One of the issues that have been on my mind lately is the last large change in the hardware technology. There is a switch from single core processors to multi core processors.
This is already very noticeable in how software programs is developed, at least if you are using the .Net Framework. It's not an obvious and huge change, but it's all those small pieces that is being added with each new version of the Framework.
From the BackgroundWorker GUI component to all those more sever directed Async wrapper methods that are found in almost any server based call.

It seems to me as we are going to see a lot more of these small adaptations in the .Net Framework. The latest I heard was that there was going to be added multi-thread support to for each loops so that the pre-compiler automatically should be able to split a for each loop into parallel tasks that was to be executed on different threads.

However, all of that is old news. I think that is a trend we will se a lot of in the future, and we will se adaptation of both languages and framework to add better multithreading support.

However, a different trend that I'm interested in is what seems to be a rapid change in portable devices as mobile phones and other portal devices.
From being very limited devices with slow processors, very limited memory and display that is unable to produce any reasonable graphics, these devices are now very rapidly getting more CPU, memory, processing power and high-res displays. At the same time laptop PC's is getting smaller and smaller. This makes the gap between laptops and portable devices to get smaller and smaller all the time. The latest technology to really close the gap between these technologies is the SSD's (Solid State Drivers). The last, missing piece is the long promised super batteries. If those were actually hitting the marked I imagine that some real changes would be seen very fast.

I think that it is very possible that we within few years from now will see a major shift in how these devices is used. I don’t expect any revolutionary, but a huge increase in the current trend of smart-phones and applications for these devices. And as this shift in how hardware is used, we will again see that the software industry will have to adapt and adjust the software solutions accordingly. Still, the change does not have to be that dramatically. All the current changes with Smart Clients, online-offline support for almost every application and other changes is all very adaptable and directed to this change.
So, I think we’re on the right track… at least until something totally different happens instead ;-)

TEK

Microsoft is releasing the source code for .Net

2007-10-04T00:36:00.000-07:00

It seems as Microsoft is planning on releasing the source code for the .Net Framework.

According to this blog: http://weblogs.asp.net/scottgu/archive/2007/10/03/releasing-the-source-code-for-the-net-framework-libraries.aspx

This is very interesting news, and it might possible be of huge help for developers. Especially those working with the more “not so well documented” parts of the .Net Framework.

At the same time I’m also a bit skeptical as this will enable the developers to look at the .Net framework more as a white box then a black box, and unintended writing code that is dependent on the inner parts of the .Net Framework to work as intendet.

Maybe you find out that some return value only may be two out of tree values, and you then code for that. In the next release the third value is supported as well but our code is not supporting it.

All over I think this is an advantage, and I assume that it is only in very special troubleshooting scenarios you will be interested in doing this.

TEK

Visual Studio 2008

2007-09-07T10:00:00.001-07:00

I was at MSDN Live in Trondheim yesterday. One of the sessions was about Visual Studio 2008 Team Systems.

It seems like MS is putting some quite nice stuff into this product. They have for example started to add support for code metrics in to the Team edition of VS 2008.

Metrics is a well known method of analyzing code and code complexity based on statistics extracted from the code. Typical information gathered is reference counters, lines of codes, number of ifs, loops and so on. Based on this information it's possible to start making some theoretical guesses about the quality of the code.
VS 2008 Team System now has added a client tool that does this and displays this information to the developer. This can then be used to evaluate the complexity of the code and give some ideas about areas where refactoring might be wise.

I do absolutely not think that this is a golden bullet for application architecture or code quality. However I do believe that it might help to put focus on limiting complexity in code and coupling between modules.

I'm quite excited about getting VS 2008 Team System installed and do some more evaluation of this and how it might work in real life.

I have started working at Fundator AS

2007-09-07T09:21:00.001-07:00

I have recently changed my work location from Norconsult Informationssystemer AS to Fundator AS.

Fundator is small, emploee-owned, consultant company. We are currently 20 employees with deep knowledge within the fields as business development, enterprice architecture, .Net and Java system development. This company has a lot of very skilled people, and I hope to learn a lot here. I'm looking forward to see what the future will bring.

There is not enough time

2007-06-06T12:56:00.001-07:00

In the last couple of months I have done some attempt to stay on top of stuff and get new inputs from sources as IT-related newspapers, blogs, MSDN and reading some books.

I really wonder if I’m a totally slow reader, using way to little time on stuff or what. It’s just so much stuff out there that I should have loved to read, understood, tested and commented upon. However, there is really-really not time in my life to keep on track with all of it.

And that’s even if I’m just sleeping 5 hours a night and drop all social life.

Maybe that also some part of the problem, I’m actually trying to see my two children (and my wife) a couple of hours every day instead of sit down in my office and staring into my computer screen. And yes, it was that damn thing called “work”. What was that again? Ah, yeah. That was the reason for why I wanted to stay on top of stuff. And to get the basic stuff there done takes of course 8-12 hours a day…

I see that my RSS feed on “Recent Blog Postings (MS Architecture)” currently has 249 unread posts, and I’m pretty sure that at least 80 of those are really interesting stuff!

But hey, after one week away from the computer I’m actually up to date on several of the other RSS feed’s I’m monitoring…

I think I’ll get some more jogging/swimming/sporting/mounting walking under my feet the next couple of months and start studying speed reading or something like that J

For everyone out there contributing with all this great blogs and discussion/stuff, thanks!
Yes, I really mean it. There is so much great reading material out that there is right to the core of important issues that it’s really, really cool! If there was only some more time… What about introduction 48 hours days?

Communicating with the customer

2007-06-01T16:00:00.000-07:00

Over a number of years I have been working with a number of different customers while planning, designing and implementing systems.

Normally these customers have been people with great skills in their work, but with little or no experience in describing software systems.

In the process we have been using a number of different tools to try to communicate better with the customers. What I have often noted is that if you try to tell the customer that “we are using use-cases as a method of describing what the software should do” or “we are going to develop a requirement specification to describe how the system is going to work” the customer often seems to be distracted by the terms used. The customers is often willing or even interested in using the methods suggested, but often it seems to meet practical issues when trying to actually do this.

For me it seems as just introducing these terms and methods by names in an early process will by itself often cause the customer to feel unsecure or unfamiliar with the techniques and methods used when describing a system.

Due to this I think that you not necessarily should introduce naming for stuff in the early stages of a project. If you for example want to discuss and show the customer a use case diagram you should just show it to the user and discuss the content of the diagram, what the arrows means and so on.

Then, when the customer after some time actually discovers that the given type of diagram is coming up again-and-again you may fulfill his felt need of giving the diagram some name, for example “use-case-diagram” ;-)

My point here is that, when working with people that is not in the industry, we should and must use our tools (their good tools) by we should also be very careful about how we approaches the customer when discussing the issues at hand. I think it is important to keep the focus on the issues that the customer is familiar with and avoid introducing terms and words that adds confusion to the communication.

.Net 3.0 is doing more for leaky .Net applications

2007-06-01T14:49:00.000-07:00

After a bumpy start with .Net 1.1 (1.0 does not count) where several issues regarding memory consumption and especially releasing memory .Net seems to be going in the right direction.

.Net has always been memory "hungry". The ide seems to be based on "unused memory is doing no good to anyone".
So, problably based one something like that, MS seems to have decided that any .Net application should grap a really good chunk of memory if available.

And that idea is resonable enough, in some closed single user fantasy world!

It soon became clear that altough .Net 1.1 applications was great at hugging memory, they were not so great at freeing it. Basically, whatever memory a .Net 1.1 application had got it's gready fingers on, was lost forever (or at least until the app closed down). In a multi user envirment with a lot of users running their applications over a long time this behaviour would very fast lead to out-of-memory exceptions as the first 20 instances of the application has already used all available memory.

In .Net 2.0 this seems to have improved a great deal, and from what I can see the behaviour and freeing of memory is mutch better. Now an application actually seems to be able to release back some memory without shutting it down first.

Now, in .Net 3.0 it finally seems as MS have understood that it is not enought to say: "Hey, we have a managed language and a garbage collector"

You will still have memory issue if you do the wrong stuff. One of the sources for doing the wrong stuff is event handlers.

Whenever you start listening to event from a event source using += you are actually creating a reference to that object. If you do not remove that reference, the event subscriber will be kept in memory as long as the event source is alive.
If you for example hooks up to the Application.Idle event in a form, that form will not be available for the GC as long as the application is executing.

On of these issues that is really important to consider is notification enabled collections. For example collections that implements IBindingList or just implements change events.
Say that you have one such collection with 1000 items within. Then you create a reference to one of those 1000 items.

For a "normal" collection, the collection has a reference to all items within the collection, the items in the collection has no reference to the collection. As soon as you no longer holds on to the collection itself, the collection and the 999 items that you are not refering to becomes available for the GC.

If you have an event enabled list, the list must to be able to notify any listeners of changes in any object within the list. To do this the list has to subscribe to events from all the entities contained within the list.
Using regular events this will create a reference from the objects within the collection to the collection itself.

So, that single object in the list that you made a reference to will now also causes the list itself, and therby all the 999 other objects to be unavailable for the GC.
This is of course not very nice when it comes to memory consumption.

I soon discovered this when working with the business layer of ByggOffice. To handle and avoid this situation I made a couple of specialized interfaces and a managment class that used the WeakReference object to support multicast events. The weak references made it possible to dispatch the events and at the same time make ureferenced entities available for the GC.
This solution worked out very well.

Now, in .Net 3.0 MS has finally started to take this issue serious and have introduced stuff as the WeakEventManager class and IWeakEventListner interface.
They are also doing a lot of referals towords the "weak event pattern":

"In typical applications, it is possible that handlers that are attached to event sources will not be destroyed in coordination with the listener object that attached the handler to the source. This situation can lead to memory leaks. Windows Presentation Foundation (WPF) introduces a particular design pattern that can be used to address this issue, by providing a dedicated manager class for particular events and implementing an interface on listeners for that event. This design pattern is known as the WeakEvent pattern."

You can read more about the "WeakEvent pattern" here: http://msdn2.microsoft.com/en-us/library/aa970850.aspx

MS Application blocks and software factories

2007-05-02T11:01:00.000-07:00

For some years now Microsoft has been giving a number of application blocks, for example the Composite UI Application Block (CAB). They also have software factories to make the application blocks easier to use from Visual Studio. One of these factories is the Smart Client Software Factory (SCSF)

It seems like the application blocks is getting more mature these days then before. It has finnally been created a decent whitepaper for the CAB and the SCSF.
It's not written by the same people that wrote the application blocks, something that problably is good.

However, personally I'm still on the sceptic side regarding the usage of these application blocks in real-life applications. Most of the application I have been involved in developing has a quite long expected lifetime, maybe up to 10 years, and several years development time (small team).
If you then look at the history of for example the CAB, you would have needed to do a lot of hard refactoring to follow up the new versions of the CAB and SCSF. At the same time the software factories will have a lot of influence on when and how you handling changes in the development environment, for example the transit from VS 2003 to VS 2005, and now coming up, the transit to Orcha.

My current stand on stuff is that you should be sure you know what you do if you choose to use for example the CAB and SCSF in your projects.

Anders Norås have a great post on the topic, and he recently also included the source code for the example project he is referring to in the blog.
It's recommended reading!

No more memory leakage, part I

2007-04-06T10:40:00.000-07:00

When switching from C++ to C# one of the thing I was glad was gone was the need of always having to be aware of memory leaks caused by forgetting calling delete() and similar stuff.
During the development of the ByggOffice application, I soon found out that this was not completely true. Even in managed .Net code you still have to watch out for situations that will cause memory leaks.

Do we have memory leakage in .Net applications?
A very simple way to start leaking memory in a .Net windows application is to just do what almost all examples shows you to do to handle UI state:


protected override void OnLoad(EventArgs e) {  
  Application.Idle += new EventHandler(Application_Idle);
}
void Application_Idle(object sender, EventArgs e) {
  // do some processing to check and update UI state
}

The tiny little detail that is let out here is that this will in effect add a reference from the form to the Application object. And as you probably would guess, the Application object will not go out of scope as long as your application is running.

This will cause the form to stay in memory as long as the application is running. And not only the form, but all objects that the form is referencing as well.
And this goes even if you call Close, Dispose or whatever you like on the form. As long as it’s holding on to that reference to the Application object, it will not be collected by the GC and it will not free the memory used.

In a real-life application this may very well mean that the form will consume several MB of memory each time it is shown. And each new instance of that form will consume the same memory all over again.

But can’t it be solved?
Yes, it can.

As listening to the Application.Idle event is actually a very useful way of updating the UI state you should still continue to use this method.
The fix for this issue is actually quite easy; just remove the event handler in the Dispose method of the form like this:


protected override void Dispose(bool disposing) {
  Application.Idle -= new EventHandler(Application_Idle);
  if (disposing && (components != null)) {
    components.Dispose();
  }
  base.Dispose(disposing);
}

The problem here is not to actually write the code, it’s to remember that you have to write the code and that this is not handled just because you are writing in a managed language.

And BTW: that other small issue:
Just thought I should mention this while I’m onto it.
Calling Close on a form shown using the ShowDialog() method will NOT dispose the form, and if the form is not disposed the Dispose method will not be called and the form will again be left in memory as long as the application is running.

This means that you have two options.

Ensure that anyone using your form as a dialog calls it like this:
using(MyForm frm = new MyForm()){ frm.ShowDialog(this); }

Move your event disconnection method to a different place, for example into the OnClosed method like this:
protected override void OnClosed(EventArgs e){ Application.Idle -= new EventHandler(Application_Idle); }

In theory the user should be using the using() approach as it will ensure that the forms Dispose() method is called in all cases and that any nessesarly deallocation done by the form is called.

However, in the real life that is not how stuff is working and depending on others to do this is a long shoot that I would not recommend.

Hello, and welcome to my blogspace

2007-04-06T09:44:00.000-07:00

In this blog I will write about different stuff that I'm conserned about.
Initially I think that this blog mainly will be regarding developer spesific stuff and not so mutch about personal stuff like my family. Maybe that will be added in a separate blog later on, we'll see.

I really wonder if anyone is going to be reading this stuff. If not it will at least work as my own personal diary regarding development technics and simular stuff.

Happy reading!