Check out this new blog I found written by a member of the Big Data community...
http://toastingit.blogspot.com/
Monday, October 12, 2009
Saturday, December 20, 2008
Agile Datawarehousing Book
I just received my copy of Agile Data Warehousing by Ralph Hughes from Amazon.
At last, someone has written this book! I hope it is good.
One of the problems we face moving the world of agile beyond objects is that and managers in conservative companies need confidence that a different approach is tried and tested. This book has the potential to help do just that.
Since it's Christmas holiday season, I hope to have some time to read the book over the next couple of weeks and publish a review on this blog.
At last, someone has written this book! I hope it is good.
One of the problems we face moving the world of agile beyond objects is that and managers in conservative companies need confidence that a different approach is tried and tested. This book has the potential to help do just that.
Since it's Christmas holiday season, I hope to have some time to read the book over the next couple of weeks and publish a review on this blog.
Monday, July 28, 2008
Naked Agilists Conference Preview
The new Naked Agilists podcast is now available at http://www.nakedagilists.com/
It's been a little while since we have released anything new, but with the Agile year's main event approaching fast we thought it would be a good idea to get some of the Agile2008 conference organisers to tell us about what they have planned. First up we have Grigori Melnik who gives us a 15 minute overview of the conference as as a whole as well as an insight into the selection process introduced this year. This year's conference is organised into stages - rather like a music festival - so we then we have 5 individual, shorter, talks from the organisers of some of the stages who tell us in much more detail what their stages have to offer.
It's been a little while since we have released anything new, but with the Agile year's main event approaching fast we thought it would be a good idea to get some of the Agile2008 conference organisers to tell us about what they have planned. First up we have Grigori Melnik who gives us a 15 minute overview of the conference as as a whole as well as an insight into the selection process introduced this year. This year's conference is organised into stages - rather like a music festival - so we then we have 5 individual, shorter, talks from the organisers of some of the stages who tell us in much more detail what their stages have to offer.
Sunday, April 27, 2008
It's much more than throwing out the documents
I was talking to a fellow Ab Initio developer the other day and he asked me a common question...
Change Control and Risk Logs
Honestly, does anyone enjoy working through these?
Requirements and designs go out of date on waterfall projects, so they need a whole extra level of documentation and bureaucracy to track changes and uncertainty.
Agile teams simply do not need any of this stuff because they are always working on immediate priorities and the chunks of work are small enough that they can actively address risks early in each iteration.
Look for the Principles
Of course, I have really just scratched the surface, but if we look for the principles behind the practices we find ways to improve our thought processes.
Documentation is a legitimate form of knowledge capture, but it's not always the best option. There are times when documents are necessary and useful, recording business metadata is an example that springs to mind, but next time someone asks you or your team to write a document, recognise it's a default position – the lowest common denominator of project communication – and everyone involved might find a better way of working if they were to kick the problem around for a while to address the root problem more effectively. Sometimes you will find a document is the best solution and other times you will not. In either case, you will reach a well considered decision based on a reality.
“Isn't all this Agile stuff just a case of not writing documentation?”
It's true that Agile teams do less documentation, but it is always replaced with better alternatives. I'll put that another way because it is important. Agilists do not throw away documents for the sake of it - that would be a pointless short term fix - but we are passionate about reducing waste in our projects and we have found there are more effective ways of doing some things that every project needs and they happen not to need as much documentation. (Mary and Tom Poppendieck showed us how to use Lean principles to identify and eliminate waste in software projects, this interview is a great place to start.)
It's true that Agile teams do less documentation, but it is always replaced with better alternatives. I'll put that another way because it is important. Agilists do not throw away documents for the sake of it - that would be a pointless short term fix - but we are passionate about reducing waste in our projects and we have found there are more effective ways of doing some things that every project needs and they happen not to need as much documentation. (Mary and Tom Poppendieck showed us how to use Lean principles to identify and eliminate waste in software projects, this interview is a great place to start.)
This can be a little difficult to imagine if you are used to waterfall projects, so let's look at a couple of typical activities on data projects and the practices we can use to do them more effectively and reduce the documentation overhead.
Test Procedures
Waterfall teams write test procedures to record the manual tests that need to run against each release. There are problems with this approach: the tests take a long time to run manually, they are subject to human error, and because natural language is imprecise and ambiguous technical details gets lost in translation by the tester.
Agile teams use Test Driven Development and executable specifications to create automated test environments that can be run quickly and easily by anyone on the project (including stakeholders). Automated tests are absolutely precise – they either pass or they fail, and if a bug occurs you can simply add a new test and that error can never happen again.
Requirements and design
In Waterfall, business analysts use requirements documents to capture what the customer needs, then the designers create another document that interprets the requirements in technical language. Again, there are two problems with this. It takes a long time, so requirements go out of date and because natural language is subject to human interpretation the customer's original need gets lost in translation.
Agile teams are cross functional including business analysts, designers, developers and testers all of whom engage directly with an empowered representative from the business. They work in short iterations of a month or less that deliver solutions to the most pressing business need at that time. There's no need to maintain detailed requirements definitions or design specs because the customer is able to communicate what is needed face to face and the tests accurately describe what the code does.
Test Procedures
Waterfall teams write test procedures to record the manual tests that need to run against each release. There are problems with this approach: the tests take a long time to run manually, they are subject to human error, and because natural language is imprecise and ambiguous technical details gets lost in translation by the tester.
Agile teams use Test Driven Development and executable specifications to create automated test environments that can be run quickly and easily by anyone on the project (including stakeholders). Automated tests are absolutely precise – they either pass or they fail, and if a bug occurs you can simply add a new test and that error can never happen again.
Requirements and design
In Waterfall, business analysts use requirements documents to capture what the customer needs, then the designers create another document that interprets the requirements in technical language. Again, there are two problems with this. It takes a long time, so requirements go out of date and because natural language is subject to human interpretation the customer's original need gets lost in translation.
Agile teams are cross functional including business analysts, designers, developers and testers all of whom engage directly with an empowered representative from the business. They work in short iterations of a month or less that deliver solutions to the most pressing business need at that time. There's no need to maintain detailed requirements definitions or design specs because the customer is able to communicate what is needed face to face and the tests accurately describe what the code does.
Requirements traceability comes for free because Agile teams never let requirements and code diverge. Put another way, the V model, never opens out into a V.
Change Control and Risk Logs
Honestly, does anyone enjoy working through these?
Requirements and designs go out of date on waterfall projects, so they need a whole extra level of documentation and bureaucracy to track changes and uncertainty.
Agile teams simply do not need any of this stuff because they are always working on immediate priorities and the chunks of work are small enough that they can actively address risks early in each iteration.
Look for the Principles
Of course, I have really just scratched the surface, but if we look for the principles behind the practices we find ways to improve our thought processes.
Documentation is a legitimate form of knowledge capture, but it's not always the best option. There are times when documents are necessary and useful, recording business metadata is an example that springs to mind, but next time someone asks you or your team to write a document, recognise it's a default position – the lowest common denominator of project communication – and everyone involved might find a better way of working if they were to kick the problem around for a while to address the root problem more effectively. Sometimes you will find a document is the best solution and other times you will not. In either case, you will reach a well considered decision based on a reality.
Saturday, April 26, 2008
Something In Nottigham
The Nottingham Agile group has a new page at http://somethinginnottingham.blogspot.com. You can use it to find out about upcoming meetings and link to the group's wiki from there.
Thursday, February 28, 2008
Agile at Dundee University
A few months ago, I was fortunate enough to bump into Janet Hughes who had the dubious pleasure of teaching me Software Engineering back when I was a funny looking undergraduate at The University of Dundee. We exchanged a few email and, on Monday, I had the great pleasure of giving a talk on Agile Data Warehousing to Janet's 3rd year students and then to the local BCS group.
There have been massive changes since I was last there. Not least the fact that the Department of Applied Computing, as it was, has moved to a fantastic new building and become the School of Computing. The range of courses has expanded to include degrees in E-commerce Computing, Computing with Electronics and Computing with Interactive Media Design, along with the Applied Computing degree in which I graduated in 2000.
The software engineering course has changed radically as well. Janet clearly has a great feel for Agile and a passion for teaching it. She told me how she discovered Agile through her own reading, was immediately taken by the way it is tune with the way that we really develop software - as opposed to waterfall; which isn't. After attending a Scrum Master course to learn more, she has added a very strong Agile focus to the Software Engineering course I took all those years ago. While I was there, her students got their marks back for a group project assignment. Working with a real customer to write a scuba dive planning system for a mobile phone in C#, they used Scrum to manage their projects, co-located their teams and developed the code using pair programming and test driven development using nUnit. Part of the final submission was a report from a Continuous Integration system one of the 4th years was writing as a final-year honours project.
I am very impressed! It is great to see a university department teaching Agile so successfully and I think it is encouraging for the development of Agile that students are starting to come out of universities equipped to work effectively on Agile teams and influencing their future colleagues and managers to adopt better practices.
Both talks went well. I was impressed by the students' knowledge of Agile practices when I ran a short discussion at the start of the talk in which I asked them to suggest practices that support each statement on the Agile Manifesto. For example, one group identified working with a product owner as an example of a practice that values individuals and interactions over processes and tools. One of the students asked a great question about how to organise multiple teams working together on the same code base.
The second talk to the BCS was quite a different audience. I was pleased that there was a good turnout for my talk – especially since the talk was at 5.30 on a Monday evening! Many of the attendees professed no Agile experience at the start, but they asked lots of good questions showed a particular interest in Test Driven Development and if/how to integrate Agile with the wider process frameworks like Prince2 you tend to find in large organisations.
I wanted to say thanks to everyone I met for a fantastic day and for looking after me so well. Hopefully I will get the chance to come back again soon!
There have been massive changes since I was last there. Not least the fact that the Department of Applied Computing, as it was, has moved to a fantastic new building and become the School of Computing. The range of courses has expanded to include degrees in E-commerce Computing, Computing with Electronics and Computing with Interactive Media Design, along with the Applied Computing degree in which I graduated in 2000.
The software engineering course has changed radically as well. Janet clearly has a great feel for Agile and a passion for teaching it. She told me how she discovered Agile through her own reading, was immediately taken by the way it is tune with the way that we really develop software - as opposed to waterfall; which isn't. After attending a Scrum Master course to learn more, she has added a very strong Agile focus to the Software Engineering course I took all those years ago. While I was there, her students got their marks back for a group project assignment. Working with a real customer to write a scuba dive planning system for a mobile phone in C#, they used Scrum to manage their projects, co-located their teams and developed the code using pair programming and test driven development using nUnit. Part of the final submission was a report from a Continuous Integration system one of the 4th years was writing as a final-year honours project.
I am very impressed! It is great to see a university department teaching Agile so successfully and I think it is encouraging for the development of Agile that students are starting to come out of universities equipped to work effectively on Agile teams and influencing their future colleagues and managers to adopt better practices.
Both talks went well. I was impressed by the students' knowledge of Agile practices when I ran a short discussion at the start of the talk in which I asked them to suggest practices that support each statement on the Agile Manifesto. For example, one group identified working with a product owner as an example of a practice that values individuals and interactions over processes and tools. One of the students asked a great question about how to organise multiple teams working together on the same code base.
The second talk to the BCS was quite a different audience. I was pleased that there was a good turnout for my talk – especially since the talk was at 5.30 on a Monday evening! Many of the attendees professed no Agile experience at the start, but they asked lots of good questions showed a particular interest in Test Driven Development and if/how to integrate Agile with the wider process frameworks like Prince2 you tend to find in large organisations.
I wanted to say thanks to everyone I met for a fantastic day and for looking after me so well. Hopefully I will get the chance to come back again soon!
Saturday, February 23, 2008
Book Reviews Kick Off
My Agile/Technical/Consulting bookshelf is coming along nicely.
In fact, it was my birthday last month and I took the unprecedented step of asking people not to buy me any more books. I reckon I can make time to read about one a month so I'll be doing well if I finish this lot before next Christmas!
Reviews
As I pointed out in my previous post, most Agile technical literature is focused on software development using Object Oriented languages and I think it's important for the development of Agile that we publish research into how to apply this vast resource of knowledge and experience to Data Management.
Now I have an influx of new books, I think now is a good time to start publishing reviews. I expect the format and structure of the reviews the evolve as I learn what works best but I think it will be helpful to the reader if I were to give each book a headline score that is easy to digest...
Let's see how we get on with that. I'm off to review a book now...
I have always been a bit of a book hoarder, and being introduced to Agile was more than enough encouragement! Christmas 2007 was especially fruitful since I pointed everyone in my family at my Amazon wish list and I got a nice little haul.
There are also a couple more titles that didn't make it in time for Christmas...
- Patterns Of Enterprise Application Architecture - Martin Fowler
- Domain Driven Design - Eric Evans
- The New Turing Omnibus (66 excursions in Computer Science) - A.K. Dewdney
In fact, it was my birthday last month and I took the unprecedented step of asking people not to buy me any more books. I reckon I can make time to read about one a month so I'll be doing well if I finish this lot before next Christmas!
Reviews
As I pointed out in my previous post, most Agile technical literature is focused on software development using Object Oriented languages and I think it's important for the development of Agile that we publish research into how to apply this vast resource of knowledge and experience to Data Management.
Now I have an influx of new books, I think now is a good time to start publishing reviews. I expect the format and structure of the reviews the evolve as I learn what works best but I think it will be helpful to the reader if I were to give each book a headline score that is easy to digest...
- Overall Score - how much did I get out of the book. Ranges from 1 to 5
- Relevance to Data Management - how much of this book is relevant to people on data management projects. Ranges from 1 to 5.
- Technical vs Management - 1 pure management, 2 bit of both, 3 pure technical
Let's see how we get on with that. I'm off to review a book now...
Subscribe to:
Posts (Atom)
