Some results of the introduction of Scrum with a maintenance projects

16. Januar 2009 by KarstenSchulze

Posted by Karsten Schulze

The most important result was the improved motivation of the team. You know a highly motivated team wants generate better results. The team was not disturbed any more by changeable requirements. At the end of the sprint we were able to provide between 95 and 120 percent of the committed scope. This was a very big success. The product owner knew what he will get at the end of the sprint.

But the product owner was not contented about all things. In particular he was not able to change the scope when the business value has been changed during the sprint. In opposite to that, I think this is the most important advantage of Scrum and not a disadvantage. But we have moved our position and we have reserved a part of our development capacity for additional work which comes up during the sprint. The consequence was that we have committed to a smaller scope, but the product owner was more satisfied. Now he was able to change the scope partly during a sprint.

Introduction of Scrum method for maintenance projects

16. Januar 2009 by KarstenSchulze

Posted by Karsten Schulze

I have heard from many people the introduction of Scrum does not make sense for maintenance activities. The duration of a small project and the fixing of several bugs are very small work items which take only 2 or maximal 3 days. Often, only one developer is involved. This is true, but you can take another view on this issue.

Please remember what Scrum means: “Scrum is an iterative incremental process of software development commonly used with agile software development.” The product will be improved incrementally. If you fix bugs if you implement new features, the product is improved sprint by sprint.

On account of the special situation in our development group I have had the idea to introduce the method for our maintenance project. We have had not enough resources to fix all bugs in the short time. We had too much bugs with the same priority. Sometimes the priorities of bugs were set wrong, at least wrong according to the business value. Finally we have to define which bug is more important than another bug. We have put all bugs and projects into a list (release backlog), which was prioritized by a product owner. By this way we have ensured that the most important bugs are solved first.

In the following article I will present some results of the introduction of Scrum.

From source code to live deployed binaries

10. Dezember 2008 by Gerhard Herbort

How does our source codes find it’s way to the AutoScout24 live platforms?

There are five stages:
1) Initial Stage –Development of features or bug fixes
2) Continuous Stage – Acceptance of features or bugs
3) Quality Assurance Stage – Verification of features and bugs
4) Rollout to Reference – Preparation of rollout an technical tests
5) Rollout Stage – The actual rollout to a couple of hundred web servers

1) Initial Stage
The way from source code to live deployed binaries starts at the Developer PC. After a developer changed the sources to fix a bug or add a feature he tests the changes on his local machine. After this test is successful he commits his changes to the source control system.

2) Continuous Stage
The first stage is the “Continuous” test system. It watches for changes on the source control system and if there are changes a new build is triggered and afterwards it deploys the binaries to the continuous test web server. There, the initiator of the change, can accept the right implementation. After the acceptance
the changed sourcecode moves to the next stage.

3) Quality Assurance Stage
The second stage is the “Quality Assurance” test system. This system is build on demand when a “Release” is schedule to go live. After building, the sources are deployed to the “Quality Assurance” test web server. At this system not only the initiator of an change has to verify this, the “Quality Assurance” team also run Smoke tests on this system as well. After the tests are successful only the approved binaries are going to the next stage.

4) Reference Stage
The third stage is the “Reference” test system. This is a web server in the live zone which cannot be accessed from the www. At this stage there some technical tests are running, eg. smoke- or stress tests. If these tests were successful the changes are deployed to the live web server.

5) Rollout Stage
The last step to live is the rollout of the changes to couple of hundred Webserver. Now the changes are live. Now the initiator of the changes make a last test if their requested changes are on live.

TeamCity - Replacement for Hudson

10. Dezember 2008 by andreaskuttruff

In the mid of 2008 Autoscout24 decided to replace the home grown solution of a “Continuous Integration” tool. As an interim solution Hudson was choosed as replacement, but Hudson has not that much features to support the developer in his work. I evaluated TeamCity and the decision was to use this tool for “Continuous Integration“.

TeamCity supports in an ideal way the established build process. We are using a build configuration chain to build and test our code. We have 3 dependend build configurations. One is building and testing the codebase. The next one builds the code in “Release” mode and deploys the result to a share. The last one uses this result and deploys the whole application into an Subversion repository. This repository is the entrypoint for deploying the application to all web servers.

Why a wrong source code structure can hurt your release strategy/development - Part I

10. Dezember 2008 by aschiessl

We at AutoScout24 deceided in 12′06 to migrate all our source code from CVS to Subversion. Back then we created a structure like this:

Old structure

Old structure

If you ignore the usual stuff, like “tags” and “branches”. You see we have trunks like

  • “release” - the trunk for the release code,
  • “development” - the trunk for continuous integration,
  • “labs” - playground for experimental stuff,
  • “docs” - random documents which need to have a version,
  • “dbscripts” - our db script trunk and
  • “deprecated” - our “fear not, its still available” trunk for people who don’t get the concept of revision control.

To get to the point of the title we ignore why we created most of those trunks. We conctrate on the two trunks “release” and “development”.

  • “release” - our trunk for release code.
    Why a separate trunk: We deceided back in ‘06 that, because of our unexperienced developers to not use the usual concept of a normal trunk. We wanted to keep our code clean, only “release” related code should be introduced into release code. That’s why we also deceided that this trunk should be “read only” to the ordanary developer. Only “qualified” personel in person of a “Release Engineer in Charge” (REiC) should write changes into that code.
  • “development” - our continuous intergration trunk.
    We deceided to make “development” trunk, based on the “release” every time we have released a version of our products successfully. If you will, we used a reverse “release” branching. Not the “release” branch is created when we want to do a release. The “development” is branched/copied, when a release was successful. This trunk is open to everybody, so they can fix bugs, add new features, aso.

So you ask, how did you get the changed code into the “release” code. We invented a thing called “Merge Requests” (MR) - a form that described what to merge. The source path, the target path, the revision range and the commit comment. These MRs were grouped by releases and they were executed by the REiC when a release was coming up.

Ok, how we did with this “unconventional” system. Fairly good, actually. For a long period of time our products were in stabalisation phase, no major features were added. And the small projects added, didn’t have a big impact on our products. So our reveresed MR based branching system worked OK. But - there’s allways a but - when we started adding larger features, this system started to fail. All of a sudden things went wrong. Merges produced a massive amount of conflicted files. Old code was replaced and reintroduced by other developers. Simply, because the developers didn’t know from which trunk to branch or they simply forgot to do forward merges, to update their user/feature branch. And to make matters worse, since there was no CI on the release trunk, problems didn’t pop up when code was merged. They first showed up, when we started to get ready for a release.

So we had to put in extra work to fix problems introduced by a wrong source code structure. We had to fix bugs in the “release” code simply because our source code system failed. Extra work which could have been avoided by simply using the right strategy at the beginning. Which lead in the end to extra hot fixes, delayed releases and missed deadlines.

Conclusion
We deceided finally to switch to the “rest” of the world. We now use a CI integrated development trunk which will be branched into a release branch, when a release is coming up and the quality gate was reached. We know we run into other problems, but at least we avoid this unecessary extra work, we had to do with our old system.

The next part will be about the internal structure of our source code and how this can hurt your release strategy.

Sprint Review Meeting

09. Dezember 2008 by Murat Akdere

Hi,

I’m Murat and works as SoftWare Developer. I will describe here in some sentences

1. What is the Review Meeting?
2. What is the Goal of the Meeting?

the end of each sprint a sprint review meeting is held. During this meeting the Scrum team shows what they accomplished during the sprint. Typically this takes the form of a demo of the new features
(Increment of Potentially Shippable Functionality) .

The sprint review meeting is intentionally kept very informal, typically with rules forbidding the use of PowerPoint slides and allowing no more than two hours of preparation time for the meeting. A sprint review meeting should not become a distraction or significant detour for the team; rather, it should be a natural result of the sprint.

Participants in the sprint review typically include the Product Owner, the Scrum team, the ScrumMaster, management, Stakeholders, customers, and engineers from other projects.

During the sprint review the project is assessed against the sprint goal determined during the Sprint planning meeting. Ideally the team has completed each product backlog item brought into the sprint, but it is more important that they achieve the overall goal of the sprint.

Back to basics – the law of leaky abstractions

30. November 2008 by DimitarEDimitrov

Back to basics – the law of leaky abstractions

When developing web applications we use high-level web frameworks and full featured controls that abstract generating the html output for us. It is however always useful to know, that what html gets generated.
Every now that I see ASP.NET code, that uses LinkButton, which posts back to the server just to redirect to the link url, issuing two http requests for something, that can be achieved with simple html anchor.

Knowing what your framework or compiler generates was always useful. Is it an old saying, that the good C programmer knows the assembly generated by the compiler. These days seeing and understanding the html source, generated from your framework is a lot easier, so do it.

“The law of leaky abstractions” is a classical software development article by Joel Spolsky
http://www.joelonsoftware.com/articles/LeakyAbstractions.html

Web Integration Tests – with or without the browser?

26. November 2008 by DimitarEDimitrov

In this post, I would like to go in the topic of web tests – automatically verifying the functionality of a fully deployed web site.
Automating the browser has been a popular approach, which gives the benefit of testing the site, as the user sees it, including images, css files and secondary functionality implemented with javascript as Ajax.
Internet Explorer can be automated through its COM API – this is a popular approach that is being used by several open source and commercial libiraries like watin and watir.
Other innovative way is taken from the selenium framework, which uses javascript to simulate user actions, which makes it portable across browsers and operating systems.

Automating the browser there is one big problem though – the tremendous amount of web requests issued for loading the complete page, including all images, css files, ad banners and so on, which can be a big performance hit if you have a lot of tests. While it is important to verify, that everything is properly deployed, all images are found and so on, the loading of all this secondary clutter does not provide any benefit for the functional tests and just slows the execution time.
When the tests run for several hours it is very difficult to use them as a feedback mechanism about the project status. Slow tests aren’t used by the developers and at the end, aren’t used at all and provide little or no value the project.

Confronted with this problem the test engineering team at AutoScout took another approach – testing without the browser, using direct http requests. This technique appears a little more challenging to use, but cuts the tests execution time by orders of magnitude. Our http tests run for a minute and half and provide very fast and reliable feedback, which makes them very useful.

If you want to use direct http tests there are several ways to write them – for .NET you can use the MS Test library that ships with Microsoft Visual Studio Testing Edition, or the HttpWebRequest/HttpWebResponse classes in the System.Net namespace. For MS Test library you have to install Visual Studio Tester Edition on your build servers.
For java there is the HttpClient package from apache. We ended up using our own small http implementation, optimized for testing purposes.
If there is interest about it, I would gladly go into more detail about it.

I would like to point out, that using direct http test requests does not replace browser tests when comes to testing ajax functionality and deployment issues of the secondary files. Our advice would be to use direct http tests when you can and browser tests when you have to. Combining the best of both worlds has given us satisfying test coverage and feedback speed.

Happy testing!

Dependency Injection vs Encapsulation

25. November 2008 by DimitarEDimitrov

Dependency Injection vs Encapsulation

When trying to show coworkers of the advantages of Dependency Injection I often encounter the argument, that it hurts one of the major object-oriented principles – encapsulation.

Let us consider the following basic example of class that has two dependencies, created in the constructor:

public class SomeClass
{
	IWriter writer;
	ILogger logger;

	public SomeClass()
	{
		this.writer = new StreamWriter();
		this.logger = new RegistryLogger();
	}
}

Client code:

SomeClass someClass = new SomeClass();

SomeClass has direct dependencies and can’t be used without them and is clearly not testable. Let us see a implementation with dependency injection:


public class SomeClass : ISomeClass
{
	IWriter writer;
	ILogger logger;

	public SomeClass(IWriter writer, ILogger logger)
	{
		this.writer = writer;
		this.logger = logger;
	}
}

This is clearly much better, but is considered to be violating the encapsulation principle from some developers. Encapsulation principle tells us to hide the class inner workings form the user. This is important in order to allow us to change the class implementation without changing all of its clients. This is a valid point, that I respect, we all strive to the “low maintenance road” (Ayende) - the most important thing in software development.

However let us look at the client code and see if it actually breaks the encapsulation principle:

ISomeClass someClass = Container.Resolve<ISomeClass>();

What we see here is that, the client code does not even know a damn thing about the class dependencies!
We just let the dependency injection container do the heavy lifting for us and don’t worry how to create all of the class dependencies.
In practice we actually do not even have the possibility to use the constructor!
That’s true – in production code instantiating a class from the constructor would require creating all of its dependencies, which would require instantiating their dependencies and so on. It would be safe to say, that the developers can’t even use the constructor and must instantiate their objects using the dependency injection container, which is so much easier.

So the user of the code does not actually even know what kind if dependencies a class has – all of this because of the dependency injection container. So actually instead of breaking encapsulation we are driving it to a whole new level – each individual class does not get to decide what kind of object it needs – everything gets configured in the configuration of the container. This definitely does not break the encapsulation principle. It makes it hold even stronger.

As a side note I want to point out, that I am definitely against the point of view “If you let the developers to bad things, then they will”. If you don’t trust your developers you shouldn’t hire them in the first place. Treat all the of the developers as stars and they will be.

Happy dependent less programming!

The Multitasking-Trap

15. September 2008 by Joachim Gmeinwieser

Posted by Joe Gmeinwieser

Last week was one of those hell-of-a-weeks in which I had planned to achieve a multitude of tasks, and come Friday had to learn that I actually got done close to nothing (hope my Boss does not read this, but in case he does, here is an analysis of that week, so at least he can see that my analytical skills are still up to standard…).

Lies den Rest des Artikels »