Software Just in Time

That was the title of my presentation last Thursday at Agiles 2009, 2nd Latin-American conference on Agile Development Methodologies. I have designed this presentation trying to summarize what the Kanban community is trying to spread recently as a new way to manage knowledge work. 

Download the slides here to follow this post about the presentation

By the way, knowledge work was the first topic of the presentation. The initial discussion was about how software development fits with knowledge work (slide #1) and how such activities can be associated with a huge quantity of variations in scenarios and contexts (slides #3 to #6). Understanding the nature of knowledge work is the key to not get trap in trying to translate manufacturing tools and techniques unappropriately.

Context Matters 

Don Reinertsen's quote about best practices and the importance of the right context to consider them as "best" warn us to be carefull when choosing best practices just because thay are supposed to be "the best" (slide #7). Context is relevant, so Kanban allows you to design processes that fit to the context, instead of manipulating the context to fit a specific process (slide #9). 

The word "process" generates fear on the agile community. In fact, there is a reason to put this word on the right side of the manifesto. But, there is no reason to think that no process is the goal. Kanban establishes a new balance on the relationship between people and processes, establishing a way to empower people to design their own processes. Hence, Kanban is a collaborative exercise for process design and offers thinking and action tools to empower people to evolve processes by themselves (slide #11). The process is owned by people, which can connect Lean with Agile in a incredible way.

And there is more... Kanban is also a Mindset

It's not so easy to structure this mindset in a way that people can understand. I give my try, anyway. So, my way to describe the Kanban Mindset is by showing patterns, tools and the type of thinking spread by the people who is talking about and applying such ideas.  The result was a pyramidal structure like the one that I introduce at slide #14: Thinking Tools, Process Design Patterns, Capability Measurements, Collaboration and Team Model Patterns and A Culture of Continuous Improvement. 

Thinking Tools

Thinking tools guide practitioners when they are applying Kanban. System Thinking help coaches and team members to understand why they are doing what they are doing. See your work environment as a system can make you conscious about causality and effects of actions. This type of thinking, despite of being very abstract, is very powerfull. A single intervention can cause a huge influence on the system as whole. 

Once System Thinking can be viewed as the starting point, Lean Thinking is the basis. Almost every operational concern is guided by the need to identify value appropriately and make it flows throught the system, improving the process continuously and eliminating waste progressively. 

Theory of Constraints is another element on this thinking process. Once you have flow in your process, you are going to start to see bottlenecks that make it slow in certain part of your system. The Theory of Constraints establishes that your system is so strong as its bigger bottleneck. By using knowledge of this theory you can dramatically leverage the throughtput of the system or, at least, you can get understanding about what are the right investments to do in your system to get higher or better results.

Queueing Theory is another key thinking tool in the Kanban mindset. What lies behind knowledge work is the unpredictable nature of arrival times and task durations (slide #19). Queues are always there and the understanding of its behaviour is in the toolbox of every Kanban thinker.

Finally, there is the Real Option theory that has been discussed lately. It establishes the basis for risk analysis by raising the importance of managing choices and defer commitments to generate options.

Process Design Tools and Patterns

The idea that processes can be designed using tools and patterns instead of procedures and documentation forms is mind blowing. In slide #21, I delineate four essential tools and some other patterns for process design using Kanban. This list is just an extract of what I have been listening as solutions to common problems used by practioners and discussed by the community. Here they are:

Essential Tools

Value Stream Mapping
Visual Management
Pull System & Single Piece Flow
Limited WIP

Patterns

Buffers & Queue Limits
Classes of Service
Leveling Work
Two tiered Systems (Expand/Collapse)
Swimlanes/Expediting
Triggers
Priority Filters
Perpetual Multivote 

Obviously, this is not the full list of patterns that people are applying to their Kanban implementation. What matters here is the idea that patterns can help a team to solve very specific problems when they are trying to design their processes. You can try to extract the basic meaning of each pattern by looking at the slides #21 to #40. You can also leave a comment on this post if you need more info about one or other.

I just want to leave a few words about the last two (Priority Filters and Perpetual Multivote). They were initially proposed by Corey Ladas as patterns for designing the process of select work to inject into the system. I think the perpetual multivote system can be particularly interesting to select work from a continuous improvement backlog queue.  

The description of those can be found here: 

Collaboration and Team Model Patterns

(slides #41 to #46)

What people call "swarming" is the most interesting pattern, in my opinion. Usually happens when people have to look at the process board and decide what to do next. The Kanban limits avoid people from inject more work into the system, so they have to find a way to help pushing the work in progress out. That's the moment when Collaboration patterns start to emerge. 

Teamlets, feature teams, and other collaboration structures start to be notice daily.

Capability Measurements

(slides #47 to #53)

I think this quote from John Seddon illustrate why we have to be carefull with measurement in our systems: "Managing with functional measures always causes suboptimization, because parts achieve their ends at the expense of the whole."

Kanban gives to the team the ability to measure capacity and use that measure to bargain or manage its commitments with customers and stakeholders. Capacity can be known using time measurements (lead/cycle time) or volume measurements (throughput). It's common for Kanban teams and coaches to use Cumulative Flow Diagrams to analyze throughput and find bottlenecks and other systemic issues on processes. 

A Continuous Improvement Culture

Finally the main goal of a Kanban implementation: establish a Kaizen culture. A culture of continuous evolution of the system. What Kanban team have been reporting so far is the constant appeareance of "kaizen events" as the time goes on. In slide #53, I mention three action tools to establish this culture: 

* doing Operations Reviews regularly; 
* enabling Continuous Improvement (CI) Actions by reserving capacity in the system to inject CI work items continuously;
* expanding oportunities for improvements removing bottlenecks, eliminating waste and reducing variability.

Last saturday I was at Agile Brazil 2009 conference. It was a great one-day conference with a lot of influential brazilian Agile thought leaders and practitioners talking about Agile. 

Interesting how Rio de Janeiro offers an audience with high mature levels of knowledge to discuss and argue about software development. It's definitely a great place to host national and international conferences about Agile.

It's also interesting to watch Lean and Kanban become so controversial in the Agile world. I can classify the Agile Communitty impressions about the "Flow Paradigm" in four categories:

1) People that believe in this paradigm as the only alternative for their projects but still don't know how to start;

2) People that believe in this paradigm as a possible alternative for very specific projects (like the ones dealing with support, bugs and intense change request).

3) People that is looking very carefully and avoiding any oficial statement until fully understand and know how it works in practice (I've noticed several thought leaders in this situation)

4) People that believe we are moving backwards with this approach.

Anyway, in my opinion, everyone should look very carefully to this new approach. After I've started to use it, I really can't see myself using the timebox approach again, even for new product development. The leverage in subordinate the system to its demonstrated capacity is really meaningfull. But I have to admit this is very risky for teams with low maturity in Agile practices.

My presentation: System Leverage in Agile Projects

The whole point of my presentation was to describe how changing your way of thinking can create real opportunities to leverage Agile projects (or even any other type of project).

I've started describing the difference between System Thinking (the new way of thinking) and Analytical Thinking (the old way). The main message was about System Thinking as a model to get understanding of why things work the way they do, in opposition to the Analytical model that gets knowledge about how things work. Use System Thinking to solve problems changes dramatically the potential to create leverage in software projects.

Some of the points of leverage that I have described in the presentation were: 

- Focus on the purpose of the system;
- Demand Management is so important as Demand Planning;
- Forget functional measures and focus on measure the capability of the system (e.g,cycle time) or the conditions of the system (e.g, failure demand vs value demand).
- Establish flow end-to-end (from the customer to the customer)
- Limit WIP
- Leadership, Team Work and generalist people are elements that improve flow; 
- Every waste activity points to the design of your process or the design of your product;
- Refactor your process is so important as refactor your code is.

You can download the presentation here:

For portuguese readers

For english readers

Last week I was in Miami attending the Lean and Kanban conference. This post contains my personal view about the conference and about some hot topics that gain my attention while I was there. 

Between Sessions on the 2nd day 

Structuring the community  

The announcement of the new Lean Software and Systems Consortium makes me think about how important is to take this community to a next level in terms of organization. I have this perception that the Lean community is fragmented today in a couple of different trends. It would be cool if this Consortium could be a real representation of the whole Lean community. But I really don´t know if it is going to be possible considering how we are structured today. Anyway, I'm an enthusiastic about this idea and I'm ready to contribute somehow to make this happen.

People

Amazing how this community is surrounded by brilliant people that really knows about what they are talking. I am still impressed with the quality of the attendees. Many of them were there to teach, not only learn. I really don't know what was better: the talks or the conversation between them. The residual learning generated by the conference came from a flat transference of knowledge, not a top-down (speakers->audience) one. The presentations were open to conversation, people asked questions during the talks, not only after them. Many times, people asked for the microphone to clarify positions and thoughts, not just for questions. The open space at the end just make this more evident.

Preparing board for the Open Space 

Enlightening People

David Anderson brought powerfull ideas on his talks. David has this ability to makes me have shining and sudden ideas when he talks or writes. It's not about trying to teach how to walk the path, it's more about direct you to a possible path and let you decide how to walk it. His recipe for success is becoming more powerfull as the community is getting more experience with Lean and its principles. WIP reduction, quality focus, demand versus throughput and prioritazation are already "a spread message" as we saw in most of the study cases. However, the new "reduce variability" message is still out of the people focus. Maybe we are going to see progress on this issue in study cases at the next conference.

An important message that I've got from David was to avoid the temptation to establish any type of "Lean Framework". Let's try to not copy the way that how other industries are applying Lean and focus on principles. If the message is strong (as it is in Lean), people are going to be able to find the right way to make it work in their own situations.

Jean Tabaka was another key personality of this conference. In her presentation, she was able to transform all the materialism that surrounds Lean with numbers, curves, graphs, and other technical stuff in something organic, system oriented. I like pretty much of the continous learning approach suggested by her. Actually, I have written about this on my article for the proceedings book. Profound knowledge about your own system, mostly formed by the business, the organization culture and the people is the most powerfull way to get leverage for growing. Jean has also an incredible leadership nature. She is always leading all conversations or activities that she is envolved. She also gives to the audience a real bonus by showing us how to facilitate an Open Space and how to conduce a wonderfull retrospective. Amazing the way that she works.

Corey Ladas was a big surprise for me. Definetely one of the most open-minded and intellectualized members of the community nowadays. You can watch him talk for hours without getting bored. I've got a copy of his Scrumban book and I have to say that the title can distract potential readers to buy it. This book is an amazing description about Lean applied to software development. A really practical view that is pretty rare to find in other Lean publications in the software area. The Scrumban technique is there, but is just a small part of a whole rich content about Lean.

Study-Case Talks

Several presentations described different ways that Kanban and Lean were introduced in a lot of scenarios. It's interesting to see how powerfull solutions are built when you don't prescribe the how ("practices and methods"), but provide the why ("values and principles"). This community is empowering people around the world to discover the best way to design processes and to influence culture organization with simple instructions like create limits to WIP and make value flows. I really like the way that Eric Willeke and Chris Shinkle prepare their material. Eric told us his story in a very unusual and interesting way. How Scrum has failed in absorb the company scenario and how kanban embraced this scenario and empowered people to make the work done. Chris Shinkle got my attention by the creative approach comparing his kanban adoption to the Dreyfus Model of Skill Acquisition. This insight is awesome not just for Kanban and Lean implementations, but for whatever initiative that intents to spread knowledge for people in organizations.    

New Room Layout just before the Open Space session 

My Presentation 

I presented at the second day just before lunch. The main challenge for me was try to describe our model, which has its own peculiar details, with the limitations of my english-as-second-language issue. But the massive feedback that I've collected after the presentation makes me believe that this was not an issue at all.

I have started my presentation describing our model which is based on a mutual trust relationship with our customers. This relationship emerges from the observance of four aggreements defined as "quick problem solving", "support for operations", "improvements for sustainability" and "new value". Actually, they are all perceptions of value by the customer (it needs and pays for all of them), but for us, three of them are perceptions of waste ("new value" is not). So they are conflicting perceptions of Value. In fact, it is difficult to establish priority between the Value generated by this activities. It's totally contextual. So we try to create balance between them. Giving to the customer nothing more than they need, at the time they need.

Classes of Services for us are the representation of this aggreements in our process. Units of work are derived from these classes of services. The Units of work are represented by cards that flows between different stages in an eletronic board that holds a set of stages distributed in 3 big areas: Demand Management, WIP Management and Delivery Management.

So, I have talked a little about each area describing some interesting issues like a prioritization filter for Demand Management, the collaboration model in WIP Management and Release per Feature and Traceability in Delivery Management.

The presentation has finished with a demo of this eletronic board and other open source tools that compose an integrated environment for technical operations.

You can download my slides here

You can read people tweeting during my presentation here

You can read reviews of each presentation here

[Update] David Anderson wrote some nice words about my presentation here

[Update] Some discussion in Agile Executive about my presentation here

So, finally, I would like to say thank you for everyone that was there and that I have a chance to talk with. I see you guys on the next conference. 

Less than two months left to the Lean and Kanban Conference in Miami and some exciting news have just happened last days.

The price is lower now and there are options for choosing between a "Full Lean Day", a "Full Kanban Day" or both. Look at the final version of the promotional flyer and open the conference web site to read more:

Using David Anderson's words: "the greatest ever assembled group of experts in Lean software development will be convening in Miami in May to explore the frontiers of agile and lean in software development". He is talking about: Alan Shalloway, Dean Leffingwell, Peter Middleton, James Sutton, Corey Ladas, Karl Scotland, Amit Rathore, Sterling Mortensen, Aaron Sanders, Rob Hathaway, Max Keeler, Linda Cook, Eric Landes, Eric Willeke, Chris Shinkle and David Laribee. He also starts to talk about them in his blog here and here.

I'll be there too. My company has been using the Kanban approach for more than a year and we learned a lot during this time. Since then, we've been building an eletronic kanban board to support the process and the way we use this board to manage our process will be the focus of my presentation.

Now we are working to detach the board from the rules of our process. We are creating features to allow process customization. The idea is customize the definition of workcells, stages, limits, classes of services, flow rules and others Kanban concepts. Depending on the acceptance of the tool and feedback from the community, we can go to the Open Source model, giving back to the community all the support that we receive to improve our processes with these ideas. That's speculative yet, but quite possible.

I see you there! 

This week an interview where I answer some question about Kanban and Lean was published on InfoQ Portal.

Check it out here:  http://www.infoq.com/br/news/2009/01/brasil-representacao-conferencia

Congratulations to Manoel Pimentel (Chief Editor of Brazilian InfoQ website) who has this iniciative to spread this work that we are trying to bring to Brazil.

There are a lot of space today to apply Lean ideas in software development, including in niches where is still difficult to stick the new Agile paradigm. Reading the interview maybe you can realize how to apply this model to common scenarios in our area.

Good reading and any feedback will be welcome.

Lean has a lot to teach for Agile practicioners. And Agile has a lot to contribute for who wants to apply Lean.

Look at this enlightening post which makes this relation more clear:

http://dnicolet1.tripod.com/agile/index.blog?entry_id=1874091

Sempre achei que toda essa filosofia ágil a qual estamos nos acostumando tinha algum modelo teórico comum. Mas sempre me pareceu estranho que este modelo fosse definido apenas por meio dos princípios surgidos com o Manifesto Ágil em 2001. A grande dúvida é: Em quê tais princípios se baseiam? São resultado apenas de experiências empíricas sobre melhores práticas ou tem algum fundamento teórico original?

Como já havia comentado no tópico anterior, recentemente tenho estudado bastante o trabalho de Jim Highsmith sobre como à partir de conceitos extraídos da teoria de sistemas complexos adaptativos é possível redefinir todas as nossas estratégias de abordagem em projetos de software. Desde então, minhas suspeitas recaíram sobre essa teoria a ponto de considerá-la a "semente original" do movimento ágil.

A confirmação veio quando li um recente post do Jeff Sutherland na qual ele esclarece exatamente este ponto. Na verdade, a motivação do seu post foi estabelecer a correta relação entre Scrum e Lean. Qual a relação entre as duas filosofias de gestão mais comentadas atualmente?

Em seu artigo, Sutherland não só confirma e detalha as bases teóricas sobre as quais o Scrum foi concebido, como também comprova a relação teórica intencional com os sistemas complexos adaptativos. Clicando aqui você terá acesso a discussões originais dele com outros membros de comunidades de desenvolvimento ainda em 1994, incluindo Robert Martin e Kent Beck (que percebe-se nesse momento já estar trabalhando na elaboração teórica de seu método).

Muito do que se sabe hoje sobre Lean teve também origem nos conceitos teóricos sobre sistemas complexos adaptativos. Ainda em 1986, Nonaka e Takeushi publicaram um dos primeiros trabalhos sobre essa nova forma de lidar com projetos no livro intitulado The New Product Development Game. O termo Lean só veio a aparecer em 1990 com a publicação do livro The Machine that Changed the World (James Womack et al), mas os princípios teóricos se mantiveram.

Em resumo, tudo o que sabemos sobre o modelo ágil de trabalho deriva dos sistemas complexos adaptativos. E estudá-los é uma boa forma de entender como e porquê eles funcionam.

Como ando recebendo alguns pedidos para envio do material que utilizei na palestra que ministrei no evento JavaBrasil, optei por publicar a apresentação aqui.

O título da palestra foi "O Paradigma Ágil". A apresentação foi baseada em uma compilação do livro de mesmo nome que estou escrevendo e na qual discuto temas teóricos que embasam os conceitos utilizados em projetos ágeis. A idéia do livro é descrever a mudança pela qual as atividades de desenvolvimento de software vêm passando desde o lançamento do Manifesto Ágil sob a ótica da teoria de "Mudanças de Paradigmas" e "Revoluções Científicas" proposta por Thomas Kuhn em 1965.

A teoria de Kuhn descreve o processo de evolução do conhecimento humano não como fator resultante do acúmulo linear de descobertas ao longo do tempo, mas, como é dito no livro "por meio do surgimento de um novo volume de idéias, percepções, circunstâncias intelectuais, possibilidades e estratégias disponíveis para especialistas de uma determinada ciência ou disciplina em um dado momento". Em outras palavras, o conhecimento humano evolui quando uma nova perspectiva pela qual determinado assunto pode ser explicado é teorizada, aplicada e provada empiricamente.

Para explicar como se dá esse mudança no contexto do modelo ágil para desenvolvimento de software, é preciso entender de onde vem cada um dos conceitos sobre a qual o movimento está embasado, como eles se relacionam e o qual o efeito esperado em se juntá-los na direção de formar um novo modelo produtivo mais eficiente.

E é nesse ponto que precisamos ser capazes de tecer toda a teia de conceitos que definem a natureza do movimento. Estamos falando de, entre outras coisas, Qualidade Total, Controle Estatístico de Processos, o Modelo Lean, Sistemas adaptativos complexos, Pensamento Sistêmico e princípios de liderança e gerenciamento modernos que juntos estabelecem a essência do paradigma ágil: a criação de sistemas adaptativos controlados pelas próprias pessoas que o compõe.

Apresentacao_O_Paradigma_Agil.pdf (485,62 kb)

Como primeiro contato com o DSDM, sinceramente achei os 9 princípios formulados por esse método - ainda em meados da década de 90 – extremamente claros e concisos, muito focados  em “business purpose” e no processo cooperativo que os métodos ágeis estabelecem. Arriscaria-me até a dizer que é uma das declarações de princípios mais próxima da essência do que é o desenvolvimento ágil.

O princípio 4 me chamou a atenção por que reflete exatamente o que eu venho defendendo atualmente:

Fitnesse for business purpose is the essential criterion for acceptance deliverables. Ou seja, o critério essencial para aceitação de uma entrega é a sua aderência ao propósito de negócio que o software endereça. O que significa que software não deve ser encarado como um emaranhado de cadastros desconexos que geram muito mais features do que uma solução de negócio para o seu cliente. Em outras palavras, ao invés de se concentrar na produção de features em massa, um time que segue esse princípio se concentrará em soluções de negócio apoiadas pelo software que está sendo desenvolvido. Isso também tem a ver com o jogo de cooperação e colaboração existente em um ambiente ágil. É como se o time se unisse para resolver efetivamente uma demanda de negócio de seu cliente, mesmo que isso envolva apresentar soluções mais abrangentes do que meramente o software produzido.

Outro momento interessante da palestra, foi quando Jean falou sobre uma técnica – até então desconhecida pra mim - para ajudar os usuários na priorização de estórias ou requisitos. Trata-se do uso do mnemônico MoSCoW.

M para Must Have (Tem que ter)
S para Should Have (Deveria ter)
C para Could Have (Poderia ter)
W para We Would Like to Have (Gostaríamos que tivesse)

Muito útil para ajudar clientes/product owners em reuniões de planejamento.

Por mais que você já tenho lido ou discutido sobre Lean Software Development, ouvir a Mary Poppendieck falar, mesmo em palestras introdutórias, é sempre um aprendizado.

O primeiro ponto que destaco é um slide que converge exatamente para o que a Jean descreveu anteriormente em sua palestra de DSDM e que eu fiz questão de ressaltar neste post. O título do slide é “Software plays a supporting Rule”  e fala sobre a característica inútil do software em si mesmo quando desassociado do seu propósito. Software só tem utilidade quando atende o seu propósito. Entender o seu propósito é a chave para construir bons produtos de software.

E qual é o propósito de um software? Ao contrário do que muita gente pensa, seu propósito não é meramente “Permitir que o usuário cadastre um...”, é muito diferente disso. Na verdade, quando pensamos em construir software estamos falando em produzir algo que efetivamente “ajude seu cliente a realizar seu trabalho”.

Essa percepção é fundamental para entender as premissas e argumentações que levam a composição das práticas em um ambiente de desenvolvimento ágil.

Um outro assunto importante tratado pela Mary, diz respeito ao nosso entendimento sobre a aplicação de controles de qualidade em um processo de desenvolvimento de software. Um bom processo de controle de qualidade se concentra em prevenir defeitos e não encontrá-los. Se você procura por defeitos, significa que eles existem. Se eles existem significa que há um problema no seu sistema de produção. Ou seja, seu sistema de produção permitiu que o defeito fosse inserido no produto.

Um outro ponto problemático é quando você aceita a existência de subprodutos sem qualidade no seu processo. No Lean, implementar significa entregar. Codificar, testar, documentar, instalar, comunicar, divulgar, publicar, etc, são atividades que definem uma implementação e que não são executadas de forma seqüencial. Tais atividades não geram subprodutos, e dessa forma, não há espera. Um testador não deve esperar o código gerado por um desenvolvedor. Ele se antecipa para trazer o controle de qualidade para dentro do processo (e não para depois dele). Trata-se de um princípio importantíssimo do Lean chamado Build Quality In, onde a qualidade é inserida dentro do processo e não após ele. 

Quando há uma área de teste que está esperando por uma funcionalidade, subentende-se que o processo de teste está separado da área de desenvolvimento e que ele recebe como subproduto uma solução de software “quase” pronta. Esse “quase” é incoerente com um processo enxuto. Não existe o “quase” nesse modelo: ou a solução está pronta (o que significa implementada, documentada, testada e outros “adas” exigidos pelo seu negócio) ou não está.  Um membro do time de desenvolvimento do produto (estamos falando de um time multi-funcional aqui), não pode partir para o desenvolvimento de um novo produto se o produto em que o time está trabalhando no momento ainda não foi entregue.