CMMI
As your company is defining the standard process based on the CMMI framework for improvement, following are some suggestions:
It is important to have the standard processes well documented, trained and used throughout the organization (New project must follow it but existing project may not have to). I want to stress the important of training to make sure everybody in your company understand the important of following the defined process. This will give management confident that your organization is on solid ground for future process improvement (Most consultants will focus on documentation and will provide samples or templates for their clients as evidences just to pass the CMMI appraisal but not much in the real result)
The most critical area is all about measurements and the application of measurements to optimize the way you build software. A solid measurement plan with clearly defined measurements for project, organization, and business and how these measures are fully integrated in alignment with organization business goals are the most important. Data, metric charts, trends must be used and posted throughout the workplace for people to see and build a “measurement culture”, what get measured will get done.
Typically when I conduct an appraisal, I look for the “culture” and “best practices”. Most organizations have a lot of best practices in place but they may not know it. “Best practices are things that work well in an organization; it could be a checklist, a template, a measurement, a training class, a method, a workflow etc. In addition to looking for document artifacts and key practices as stated in the CMMI books, a good organization must have significant data and practices not documented in CMMI and this is where I pay a lot of attention because it is the “culture”, the way people do work and operate that will really mature the organization. So I am looking for:
1. ROI and Improvement Trend Data
If an organization has used CMMI for many years, they should have improvement trend data – how much they have invested in process improvement (total efforts per software engineer) and what kind of business benefit they have obtained in terms of cost, schedule, and quality (e.g., decrease in post released defects by X% within X years, increase in customer satisfaction to X% within X years, etc.). A well-run organization would have graphics that show improvement trends over time for each maturity level against business goals and objectives.
2. Lessons Learned and Unique Practices
A matured organization also has lessons learned documented somewhere so people can access and avoid mistakes. (The best way is to have a website where people can access and contribute) I want to find out what barriers that they had to overcome. These barriers may be in process, measurement, cultural, business environment, or customer relations. The intent is to identify the things that had to be done differently to get the business results. Are there any things that they tried and abandoned because they weren’t working. I am looking for unique practices that they do now because it is their strength and skills even it is not required in the CMMI book. Maybe this is an important contribution to their business success and it can be demonstrated as high maturity capability. A highly matured organization always involves customers, most inter-group coordination always cover customer activities and this kind of team building determines the solid level 4/5. Typically, in an appraisal interviews, you will not hear about complain on management or customer but rather a significant attitude toward teaming and working together. Data on customer meetings (Meeting reports) results of those meetings are good artifacts to look into. You do not wait until level 5 for defect prevention but rather hear about them at level 4. If there are significant reviews per life cycle phase, people know how many defects are discovered and fixed them before release. Typically, an 85% or 90% defect detection rate is expected at level 4/5. As a rule of thumb, less than 80% may be a solid 3 but not quite a 4. The standard process document (OSSP) at level 4 is much more organized and refined with significant reuse and tailoring. Data on schedule variance, defects, and time are always under Statistical Process Control. Upper and lower bound are determined by historical data not just by typical standard deviation as common practice. I always ask how they come up with their Statistical Process control to determine if they are a solid Level 4 or not.
3. People and Cultural Issues
Most Level 4/5 organizations are also doing something right about people issues. Typically I find a lot of team building, people-oriented management, and skills building activities. Many have an established formal mentoring program and have a significant orientation program for new hires. Data on employee turnover is kept and known by management. Some have data on employee morale, which has improved as a result of the process improvement activities. As a rule of thumb, I also ask what they are working on next. A solid Level 4/5 knows what to do next and a “fluffy Level 4/5″ does not have vision past Level 4/5 typical answer such as achieve Level 5 by 2010 and that is it. One factor that determines a solid Level 5 is the answer to what they will do next and their improvement objectives. What kind of practices are they refining or optimizing. What barriers may they be facing and how will they overcome them?
4. Finally, I always ask people organization questions. “What does it mean to be level 4/5 or to be the top software company? What is different? Are there changes in employee morale and motivation? Are there changes in business results such as productivity and quality what about performance excellence, operational excellence, and customer satisfaction? Since Level 4 focuses on quantitative management and apply Statistical Process Control (SPC), I want to know what measures are useful. What analytic techniques provide insight? Who is using SPC, and what value are they obtaining? How does management operate at higher maturity level differs from lower levels? Do sponsorship issues change after Level 3? What are the issues in maintaining excellence after achieving Level 4 or 5?
As you have things in place, I can ask one of my staff to visit your organization for a review, if things are good then I can arrange for a CMMI appraisal.
Sources
- Blogs of Prof. John Vu, Carnegie Mellon University
