.net, microsoft, software development, visual studio

Real-world Microsoft Team Foundation Server: Not very good, says ThoughtWorks

12 Comments

I spoke to Sam Newman, who is European Continuous Delivery Practice Lead at ThoughtWorks, a software development company. Needless to say, we talked extensively about Continuous Delivery and I will be reporting on this separately; but I was also interested in his comments on Microsoft’s Team Foundation Server (TFS).  He told me that ThoughtWorks teams often end up working with it at their .NET clients, but it is problematic. In one case, he said, 6% of productive time was absorbed dealing with TFS.

What was the problem, performance, bugs, features lacking?

When we’ve looked at the problems we’ve had, a lot of it unfortunately comes down to the version control system. It’s not very good. It’s slow, you can’t do rollbacks, sometimes things go missing, you get locks. When we talked about 6% of time, they were things like waiting for a solution to expand in Visual Studio. A lot of those issues are in the version control system.

A frustration is that you cannot use TFS with any version control system other than its own.

Every other build server in the world, from Anthill to Go to Cruise Control to Hudson, you can put in at least 10 version control systems. In TFS they are all coupled. So you can’t take the version control and point it at Subversion. That might resolve a lot of the issues.

Why is TFS so widely used? It is because it comes in the box, says Newman.

I can’t think of a single client that wanted a tool, went out into the marketplace, and selected TFS because it is the right tool for them. Most clients use TFS because it comes with their Enterprise MSDN licence.

I have tried TFS myself and found it pretty good; but then I am just testing it on small projects as a solo developer, so it is hard for me to replicate the experience of a real-world team. You would have thought that performance issues, such as waiting ages for a solution to expand in Visual Studio, could be solved by tracing the reason for the delay; but apparently this is not easy.

This is anecdotal evidence, and of course there may be plenty of TFS installations out there that work very well; I would be interested in hearing of counter examples. I am also not sure to what extent the problems apply to all versions of TFS, or whether there is improvement in TFS 2010.

Newman recognizes that anecdotal evidence is not much use: he says ThoughtWorks is trying to collect some solid data that can be used both to discuss with clients making version control and build system choices, and with Microsoft.

Performance is a feature, and makes a large contribution to user satisfaction. The first release of Outlook 2007 was extraordinarily slow in some setups, and I remember the pain of clicking on a folder and then waiting tapping my fingers while it thought about expanding. It sounds like some TFS users are having a similar experience but in Visual Studio.

12 thoughts on “Real-world Microsoft Team Foundation Server: Not very good, says ThoughtWorks”

  2. @Craig I think the suggestion is not that you can’t do rollbacks, but that they don’t always work as they should. That’s how I took it anyway; I’m aware that it is a feature.

    Tim

  3. @Craig Stuntz: Using a command line only tool for an all-inclusive source control system is only usable in certain batch situations. Clearly MS did not want this tool to be as visible as it can. The question is: why?

  4. Jeff, there are a lot of things you can only do from the command line. This is true of every VCS I’ve ever tried. I’m not trying to say that TFS is the best tool ever, but ThoughtWorks seems to prefer git (which I also like). Want to see a tool that makes you use the command line? Try git.

    My $0.02 on TFS. The VCS functionality (in 2010, which is all I’ve tried) is decent, but not market leading. Works OK, though. The process management is better than anything I’ve seen.

  5. So, a big-name consultancy firm that rather likes Java and open-source stuff thinks a Microsoft product is sub-optimal? Shock horror! Whereas Subversion is so perfect that it’s not like people are leaving it in droves for Mercurial or Git… erm. In point of fact, isn’t Subversion the ‘in the box’ solution for Linux distros? The default choice? How many companies go through a full cost / benefit analysis and conclude Subversion is their source control system of choice, versus how many use it because ‘that’s what people use’? In its own way it’s just like SourceSafe used to be – ubiquitous. And ubiquity doesn’t make it good. I mean, SourceSafe… shudder. And I speak as a many-long-year user of that particular product.

    It’s like a Mac user using a PC – sure, he can use it, but he really *wants* to go back to a Mac, and anything he doesn’t 100% get will be called a ‘problem’ with the PC.

    Is TFS perfect? Far from it. Before 2010, it was a major pain to install and admin. Now they’ve fixed that, but it’s still imperfect. The Visual Studio integration does cause problems – though again, less than in earlier versions. Project files are a particular pain. That said, I’ve used it, day in and day out, since the day it was released, and I’m not about to switch to anything else. It integrates better with the MS dev tool chain than anything else.

    As for the criticism that it isn’t modular – well, what do you expect? It’s a part of Team System, it’s supposed to be used that way. You aren’t supposed to buy and use TFS on its own with NUnit or whatever (or vice-versa). If you’re a mixed-platform shop then you’re highly unlikely to be buying into the MS toolset beyond the IDE anyway, and if you’re an MS shop then why stray outside the simple-ish path that’s been created for you?

    If they can show figures that prove their productivity drops on TFS, then fair enough. But I’d like to see those compared to similar scenarios from all-MS shops that have used TFS long-term. Experience = speed + knowing how to work around problems.

  6. @Neil

    The Java comment is not altogether fair; ThoughtWorks does a lot of .NET work, otherwise I would not have bothered quoting Newman on the subject.

    I agree there are a ton of good features in TFS and personally, as I mentioned, I’ve got on fine with it; but my usage is limited.

    Not sure about the install and admin. Yes it is greatly simplified for a small-scale install where you accept the defaults; but I found it was still complex if you wanted full-features and especially thing like virtual lab management. It is not such a big deal though; considering the value it delivers it is worth a certain amount of pain.

    Tim

  7. Thoughtworks does seem to have an anti-TFS bias. In their very good book “Continuous Delivery”, Jez Humble and David Farley, both whom have worked at Thoughtworks, have a brief paragraph on p. 386 recommending everyone stay away from TFS: “Its tight integration [TFS/Visual Studio] is perhaps its only distinction. Otherwise, there is no good reason to use its source control offering, since it is essentially an inferior knock-off of Perforce. Subversion wins over TFS hands down”. This book was published in 2011, so I assume they’re referring to TFS 2010.

    We’ve use TFS 2010 internally and have found it meets our needs quite well. We’ve got about 40 C++ projects with a lot of files, and TFS speed in VS2010 hasn’t been an issue. I can’t compare to Subversion or git, but I don’t have major complaints with TFS version control (perhaps being subjected to SourceSafe for many years has made me more forgiving). The work item tracking and build management are now working quite well for us, and certainly helping us work towards “continuous delivery”. It did take some effort to configure TFS properly (especially the Sharepoint integration). I’d like Microsoft to spend more effort in making the setup configuration easier, especially for small teams (once your needs go beyond the default configuration, you kind of fall off a cliff and have to wade through a lot of details to configure everything properly). Microsoft seems to have built a good foundation, and now they need to focus more on exposing and packaging these features through more built-in UI (there’s a ton you can do with the TFS API).

    Just my thoughts – it’s certainly not a terrible product, and I’d be interested in a balanced comparison with other products that are out there.

  8. TFS Server was not always “in the box”. It is only up until recently that it became part of the MSDN subscription. I’ve been working with TFS since 2005. It is not a perfect product, but I agree with Neil Hewitt: The “Anything but Microsoft” community will always lob cheap shots at anything Microsoft. Its part of their DNA. And while there are several great options available within the ALT.NET community, I’ve seen many clients succesfully adopt TFS.

    The fact is that there is no better development environment or IDE than VS- just ask developers why they program in .NET: Productivity. So yes, Sam is right that they go hand-in-hand, and that is by design. If you don’t like TFS, use any other MSSCCI supported system, but find me one that succeeds at intersecting process and SCM and supports traceability from the task to the line of code. Sure, there is an opportunity for MS to innovate around DVCS, and if DVCS is what you are after, then TFS may not be for you.

  9. I would like to see any VCS that doesn’t cost a small amount of productivity time… I have used and use many daily(TFS, SVN, mercurial and even CVS still!.. git was in such a bad shape for windows that it had to go, it is better now though), and I have yet to find a VCS that doesn’t have issues every now and then.

    I also don’t understand the comment that you can’t use subversion with TFS build? Of course you can, just like any other CI tool you will just have to add you own checkout command to the build files..

    Also if he uses TFS so much he should be careful with the naming, TFS SC and TFS Build is not the same thing.

    I would have understood if he attacked the cost, because there is a substanial cost to TFS compared to other solutions. However those 6% productivity loss usually goes higher one such solutions in my experience.

    I think Neil’s comments are probably close to the truth of the criticism.

    On the note of VCS:s, Veracity intrigues me a bit.

  11. We’ve just moved from TFS to Git and haven’t looked back. Our TFS installation caused us no end of trouble. It’s unintuitive interface (and yes I am comparing this to Git’s), its inability to reliably “Get latest version”, phantom locking of workspaces and its tight coupling with Visual Studio (we had non VS based apps) caused friction all over the place. TFS as source control system cost us up to a 10% drop in productivity. Its task system was primitive and this was even before we looked at CI. Git, Team City and a white board are just solving all those problems for 0 monetary cost and more importantly a lot less friction.

  12. It may be worth trying out the ancient SCCS. A very decent variant of it was Sablime from Bell Labs. With a handful of commands (<6) most developers can effectively work w/ it. Its "interleaved version control" is simply amazing.

    Sexy UI, tight integration etc. are, IMHO, for the laity. For any decent programmer, these shouldn't even be a factor.

    Copied from Wiki:
    "SCCS was the dominant version control system for Unix until the release of the Revision Control System (RCS)[dubious – discuss]. Today, SCCS is generally considered obsolete. However, its file format is still used internally by a few other revision control programs, including BitKeeper and TeamWare. The latter is a frontend to SCCS. Sablime[3] has been developed from a modified version of SCCS[4] but uses a history file format that is incompatible with SCCS. The SCCS file format uses a storage technique called interleaved deltas (or the weave[5]). This storage technique is now considered by many revision control system developers as foundational to advanced merging and versioning techniques,[6] such as the "Precise Codeville" ("pcdv") merge."

Comments are closed.