While at Thought Machine an important part of my role was evangelism for product thinking and advocacy for the discipline of product design. There were two primary reasons why such activity was important:
1. The ratio of full time product managers/ product designers to engineers was low and as such we needed to cross-skill and enable engineers to play product roles;
2. We believed that Thought Machine's product quality would improve through investment in the product discipline. As a result we knew we needed to build awareness of what good product design and product management looks like.
The culture at Thought Machine put emphasis on longer-form asynchronous communication and so to take advantage of this I set about writing a short book - I called it a playbook - what follows is an abridged version.
Introduction
Understanding what product design is and what it involves can be tricky as its definition will vary depending on who you speak to or what article you read. That said at Thought Machine we propose a simple definition:
Product design is a discipline that pursues a complete understanding of client problems and crafts answers that work for both the client and our business.
Product designers do not work in isolation. On the contrary for any product development activity to be successful, it is essential that at a minimum you have co-located a product designer, product manager and tech lead. This is because any plausible product solutions will have multiple dimensions of risk that need to be continually assessed, where these different risk dimensions are best understood by the different roles:
Product risk accountability
Business viability risk - Will the product work for the wider set of internal stakeholders, such as sales or legal? Will the product be commercially successful?
- Owned by the Product Manager
Usability risk - Will the product be effective when implemented given all possible upstream/downstream use cases?
- Owned by the Product Designer
Feasibility risk - Can we implement the product in a realistic timeframe and with necessary performance/reliability?
- Owned by the Tech Lead/Technical Designer
Value risk - Will the product overall satisfy both the client’s desired outcomes as well as our own product strategy objectives?
- Co-owned by all three
Playbook purpose
This playbook will focus on detailed exploration of the activities and techniques that should be employed by a product designer in order to ensure that we are able to fully understand any client problem. Then to take this understanding forward to craft product solutions that are usable and deliver value for the client as well as our own business.
The playbook will frame these activities using a time ordered lifecycle, which the reader can use as a guide as product work progresses.
For simplicity we give this lifecycle a name, we call it Product Discovery.
Note that we will use the term product designer regularly in this playbook, however this term does not just apply to full-time product designer colleagues. Rather it applies to any colleague who needs to “play the role” of product designer as a result of a resource availability reality.
If the reader is not a full time product designer then hopefully this playbook will make clear the significant responsibility that this role carries.
Definition of discovery
The concept of product discovery is not complicated, however it is often misunderstood. All too often it is considered to be a finite piece of work mostly done by “product people”, which once complete is “handed over” to engineering. While this form of waterfall practice may be convenient, from say a planning perspective, the reality is that very few successful technology products are built this way.
Instead, we approach discovery as a continuous cycle which will be most successful when there is meaningful collaboration from the primary stakeholders - those being the product manager, the product designer and the tech lead. Where this collaboration must necessarily commence at the inception of any project and continue right through to its end.
Lessons from Gold Mining
To try to better illustrate the ongoing nature of discovery and to create a picture of how discovery should work let's pause for a moment and consider an entirely different industry, namely mining within the gold fields of the Yukon, Alaska.
In the Yukon, all of the land that is zoned for gold mining is divided up into claims, which in turn are owned by individuals or corporations. Claims are either described as virgin ground if they have not been previously mined, or mined out if they have been. However even when they have been mined out there is still a chance of residual gold, as gold recovery techniques used by the old-timers was poor in comparison to today’s methods.
As an aspiring gold miner the first task to accomplish is to secure a promising claim, what is conversationally referred to as good ground. This is the first task because the ability for the mine boss to secure a crew of workers as well as the leases for mining equipment are predicated on first having good ground. When looking for a claim there are many considerations but chief among these is whether the geology and topology are conducive to producing gold. A good example of this may be looking for signs of an ancient river bed.
Having identified potentially attractive ground it will be necessary to either buy the ground outright or to take up a lease with the landowner. It will also be critical to make sure access for equipment is feasible and that you have a water permit that you need in order to wash the rocks and produce gold. It is common for the mine boss to negotiate for core drilling samples within the claim in order to increase the certainty that the ground will pay off prior to contractual commitments. However there is never total certainty and a leap of faith will always feature to a degree.
With a claim, equipment and crew secured the mine boss next needs to create a plan for how to approach the ground. What happens next is called prospecting, where the more successful miners know that good test data can make a big difference to gold recovery results given the short mining season between each harsh winter. A tried and tested prospecting method is as follows:
1. Based on topology indicators, draw out zones within the claim that are of high interest;
2. Conduct grid pattern core drill tests within identified zones;
3. Analyse core results based on:
a. How deep was gold found? (deeper = more expensive to access)
b. How concentrated was the gold? (more concentration = better payoff)
c. How consistent were surrounding tests? (low consistency = more risk of small streak)
d. How easy is water source supply (harder = more expensive groundworks & maintenance)
4. Having got favourable core samples for a zone carry out a 1000 yard test (small pilot operation runs for 24hrs);
5. If 1000 yard test proves profitable commit to larger operation.
Of course the concerns of the mine boss do not end there. There are a host of ongoing optimisations to consider during the operation. These include how much ground to open up for the operation, the gold recovery success rate of the wash plant (how much is being kept/lost), the proximity of the wash plant to the dig site and the logistics of getting pay dirt out of the ground and through the wash plant.
Lastly beyond concern for the current operation it is clear that the ground is finite and as such there is a need for continuous prospecting activity in order to develop future options. As the size of the mine boss' enterprise grows the number of concurrent operations will also grow, which will only add to the need for ongoing prospecting work.
Mapping to discovery at Thought Machine
Hopefully you will already be forming some connections between gold mining prospecting and our own need for product discovery. To ensure clarity of meaning here consider the following mapping:
- Sourcing/selecting a claim = Product vision & high level product strategy
- Zoning within a claim = Broad assessment of client problem spaces*
- Core tests within zone = Investigating root problems & high level solutions*
- Selecting area for 1000 yard test = product prioritisation & low level strategy*
- Carrying out 1000 yard test = prototyping to assess solution feasibility/usability/value*
- Decision to commit to operation = low level solution suitability/viability decision*
- Commit to an operation = establish an inter-disciplinary team** to deliver an already validated solution.
Items marked with a * are collectively what we shall call primary discovery and are the focus of the product designer activity.
In addition to this primary discovery activity it should be acknowledged that the product designer is also a stakeholder within high level product strategy formation (claim selection). Which arguably you might call another higher form of discovery. At the other end the product designer is also concerned with the need for continued assessment/optimisation/refinement within the delivery stage.
It is important to acknowledge that foresight and solution assessment during primary discovery will be imperfect, as a result course corrections may well be required even during delivery work.
** An inter-disciplinary team should not be confused with a multi-disciplinary team. In the latter you have a structure where each team member is responsible for a particular role with other members entirely deferring to the specialist in all matters within that specialism. In inter-disciplinary teams while you continue to have role specialisms, the team rather than the individuals co-own all aspects of work regardless of specialisms.
In inter-disciplinary teams there will be a greater sharing of responsibility and empathy for the distinct roles. Through such a structure the inter-disciplinary team will on the whole exhibit more missionary style behaviours and as such will have a much better chance of achieving desirable product outcomes than the multi-disciplinary team.
Discovery lifecycle
As we have described, the product designer activity is for the most part concerned with carrying out discovery work that will yield high quality product outcomes. We have also acknowledged that discovery is ongoing rather than being a phase that is completed and then handed over.
Our gold mining analogy has given us an idea of what may be broadly be involved. The following diagram takes this a step further, proposing five specific phases within discovery as well as acknowledgement of the input made by product vision and strategy. Let’s consider each in summary:
Phases of the discovery lifecycle
Product vision and strategy
While vision and strategy formation are primarily the responsibility of the executive, with the product manager being the ongoing custodian, the product designer is well positioned to provide learned input. Both vision and strategy have a dual role of directing ongoing product work and enticing our enterprise clients to join us on the journey to fruition.
The product designer must take the time to become fully acquainted with all proposed vision and strategy material in order that she can play a key role in communicating it to clients in relatable terms. Here we are looking for two initial outcomes:
Client feedback on vision & strategy can be captured and used potentially for refinement;
Clients are brought into the plan and keen to participate in discovery activity as a research participant.
The product designer role puts great emphasis on the use of both storytelling and interviewing techniques. Within this phase of the lifecycle these skills must be leveraged to ensure that vision and strategy are on the right path.
Having cemented vision and high level strategy again the product designer must be very well acquainted with their shape and context. This is because the product designer skill set also lends itself well to acting as an internal advocate and champion for them among the wider team. Indeed such advocacy is most successful when the product manager, product design and tech lead all contribute in their own way.
Problem definition & investigation
The first of our primary discovery phases where product designer activity will ramp up quickly. A well formed high level product strategy should for the most part be written in terms of broad areas of problem which need to be addressed; what we will call a problem space. The strategy should not however get into low level details about the nature of the problem nor should it attempt to consider the solution. Rather these are the initial goals for the discovery activity.
In this phase the product designer in conjunction with the product manager are focused on unpacking the strategy to determine what is known/unknown within a given problem space. In the following section we will consider activity within this and subsequent discovery phases in more detail, for now we can thinking of this phase as being concerned with establishing the lay of the land at a fairly high level and scoping further discovery work.
Design research
Having established a base picture of the problem space and considered scope of further work, the design research phase is concerned with in detail knowledge gathering. At this point the product designer will have either established who the target user for any solution is or will have a hypothesis on who they may be. The product designer will deploy a number of techniques to gather low level knowledge but among these the most important is the 1-2-1 client interview (detail to follow).
The goal of this phase can be summarised as investigation into the root problem(s) that is (are) preventing the client from achieving their desired outcome(s). Of course in order to establish the nature of a root problem, the product designer must also extract and understand all relevant client outcomes. We call these outcomes the client use cases, which can either be directly observable within the problem space or are upstream/downstream of it.
Solution design & architecture
With initial knowledge gathering activity carried out the team should have sufficient input to fully understand the nature and shape of the problem to be addressed. Within this phase there is again a significant focus on collaboration between the product designer, product manager and tech lead. Where the collective mind is tasked with conceiving ideas of how to best solve the problem(s) in scope.
The product designer will here be using her skills to consider how the architecture of any system will impact usability either directly or within any upstream/downstream consumer. The likelihood will be that there will be give and take between the occasionally misaligned forces of usability/feasibility/viability/value. Where the product designer will champion usability and value but defer to the product manager for risk arbitration decision making.
Prototyping & validation
With the solution design phase done it does NOT mean that you can jump straight into delivery. Instead it simply means that the team has come together and formed one or more ideas for a solution to the problem. The prototyping phase is all about taking an idea and using a variety of techniques to form greater confidence that if delivered the solution would satisfy the client’s desired outcomes and work for our own business. We call this goal validation, where the principle tool employed to achieve validation is the prototype.
A prototype can take many forms and can be used to test all of the different types of risk. The product designer is predominately concerned with prototypes that can be used to test both usability and value. What is perhaps most important to understand about prototypes is that they are supposed to be quick to construct (a week or less) and for the most part be disposable. The product designer will need to determine the scope and appropriate fidelity for the prototype and orchestrate testing with target users. The product designer is also principally tasked with analysing test results and forming conclusions to discuss with the wider team.
A quick note on prototype vs. delivery milestone:
At Thought Machine perhaps due to the cultural focus on engineering excellence there is a predisposition to jump straight into delivery of production code and to refer to early milestones as prototypes. This is a mistake as the latter will in most cases take far too long to create and will also likely create sunk cost resistance and confirmation bias when it comes to prototype assessment.
A prototype should in most cases take less than 1 week to build and should always be considered throw away. As such most prototypes, with the general exception of feasibility prototypes will likely involve little or no code, and never production quality code.
Build evaluation & testing
The collective goal of the previous three phases is to arrive at a validated solution to the right problem(s). The product designer will in most cases be the principle advocate of ensuring client validation prior to commencement of delivery activity. However as mentioned earlier whilst validation activity can help grow confidence, it will never give you complete certainty. The best we can hope for with validation is that there is enough of a signal that we are headed in the right direction in terms of creating a solution that will provide a good combination of viability/feasibility/usability/value.
During delivery of the project it is likely that there will be delivery milestones that will be candidates for pre-release. In case the reader is unfamiliar, we seek to offer pre-release as a means to further evaluate and subsequently optimise aspects of a product before we become subject to the backwards compatibility rules of our API guidelines. In this phase the product designer as well as the other team members must be chiefly concerned with how best to measure the low level suitability of the solution. This phase must in no circumstances be used for primary validation, as pivoting in the case of a fail would be fairly disastrous. Instead the product designer will work with the team to formulate a strategy to test the solution with a view to then later being able to further optimise the product.
One aspect of Thought Machine’s product that has historically been entirely absent are usage analytics. While we have made some progress in observability metrics, these are constructed for the purpose of monitoring health and troubleshooting problems within an instance and not measures by which to access overall use or suitability of a product component.
The product designer along with the product manager must obsessively advocate for product assessment metrics and telemetry. Put simply we must never ship a product without the metrics by which we can measure its success. Commentators who say that such analytics are not possible within deployed enterprise software are simply not being creative enough. This is a problem like any other that the product designer will be ideally suited to solve.
Short vs. long feedback loops
When considering the proposed discovery lifecycle it is important to appreciate that in practice the cycle of work will not see progression neatly forward through each phase. In fact if you find yourself being in a position to progress from one phase to the next without issue or challenge then this is a red flag that you have probably taken one too many shortcuts.
What is typical and in fact desirable is that there will be many short loops back to previous phases based on new information. For example a solution prototype reveals deeper understanding of the problem that triggers new ideas for a more optimum solution. The product designer is absolutely on the front line of watching out for these continuous learning opportunities. Indeed it can often be helpful for the product designer to consider how facilitate the circumstances to break/disprove current assumptions.
Principles within product design
So far we have established the fundamentals of the discovery lifecycle and introduced the broad range of responsibilities that the product designer has. Before we dive deeper into product designer activity with our five discovery phases we need to consider a set of guiding principles.
Below are a set of 12 principles that are of particular importance within product design at Thought Machine.
1. Create “organisational” knowledge
While it may seem obvious as stated this principle is often overlooked because it takes considerable effort to achieve. Simply put it is not enough to arrive at individual or even local team level tacit understanding of any given topic. Rather it is the responsibility of the product team and the product designer in particular to inform, and if needed educate, colleagues as necessary concerning acquired knowledge.
The best strategies to accomplish this are to use mixed format communication with multiple repetitions. That said regardless of strategy employed the absolute non-negotiable in all situations concerning handling of knowledge is to write it down and make it discoverable. The product designer must lead from the front here and advocate for others to do the same.
At the time of writing this means creating well structured content natively within confluence within the relevant discovery workstream.
2. No substitute for research with clients
Our product at Thought Machine is for enterprise, we are also a platform rather than a destination product. These two facts mean that orchestrating and conducting value generating discovery interactions with the right people within client organisations is a big challenge. Faced with the difficulty here it is tempting to look for alternatives such as SMEs or user proxies within our colleague teams.
While there can be value in colleague interactions, the product designer must be resolute that in no way is this a substitute to high quality client interactions. Working with the product manager and client services teams, the product designer will need to adopt an uncompromising attitude here. Where the goal must be the fostering of a wide based network of knowledge holders within our client orgs that can be regularly and easily mined for discovery purposes.
3. The first problem is rarely the one that needs solving
Having created such a discovery network with our client base and had some face time, whether that be observation, interview, workshop, it will be tempting to latch onto the first juicy problem you discover. The product designer must however be cautions and diligent here. In all cases assume that all problems are in turn caused by another problem.
As such the objective when a problem comes to light is to first determine its root cause. Only then will it be appropriate to best consider at what level of the problem space will a solution have the most impact.
4. Aim for minimal subjectivity
During discovery work it will often be tempting to take a limited set of inputs and combine them with some internal logical inferences to yield the root problem, or indeed the optimal solution to a problem. It is the responsibility of the product design first and foremost to ensure that factual inputs are as complete as they can be. Furthermore where the product designer feels that input is not sufficient she must strongly advocate for caution against making assumptions that are not based largely on client originated evidence.
5. Strong beliefs loosely held
While making evidence based decisions during discovery is always the goal, the product designer must also appreciate and balance that there will come times where pragmatism will require the use of ones belief or instinct to move forward. While this is sometimes unavoidable and in some cases can create the conditions for break-out innovation, caution must be maintained.
Outwardly it will be beneficial for team if the product designer presents a strong conviction in her beliefs, however inwardly the product designer must be willing to rapidly abandon a belief if new and contradictory evidence comes to light.
There is one further perhaps inconvenient truth to acknowledge here. Put simply the likelihood that the instinctual beliefs of the product designer, or for that matter any other team member, will stand up to scrutiny and ultimately be right is directly connected to how much experience the holder of the belief has. If the product designer or the colleague playing this role does not have significant accrued experience within the area in question then even more caution should be present when it comes to progressing discovery work based on a held belief.
6. Resit the urge to educate the client
Like many startups/scaleups within fin-tech we see ourselves as the disruptive innovator that is often tasked with saving the client from themselves. This belief system is an important source of conviction but can also be dangerous. Without care we can easily find ourselves being dismissive of what we learn from client interactions. This is a tricky balancing act that the product designer must be able to recognise and course correct if needed.
As product designers we must always be aware of whether a solution is actually better for the client now or in the future or rather just something we think should be better for the client. The product designer is the ultimate champion for the client’s goals and intended outcomes within the use of our product and must be willing to step in if a disruptive solution is in fact a step too far for the client.
A tell tale sign that we have gone a step too far is where we find it difficult to make the client understand the benefits of a different approach. Put simply, if value of switching isn't obvious with little or even better no explanation then it is probably time for a climb down.
7. Recognise the feature factory threat
Within enterprise software development companies there is always significant danger that product and delivery teams can find themselves in feature factory territory. There is a human nature aspect here in that it is a path of less resistance to simply design/deliver what others tell you to. Where “others” in this context can either be our clients or indeed our own colleague teams, including senior management.
The product designer needs to support the product manager in creating a line of defence to stop this from happing. Where in this case the best line of defence is in fact offence in which all inbound solution requests are subjected to an appropriate levels of challenge. A healthy process is where the inbound requests are asking for assistance with a problem and not to ask for or propose a specific solution.
Any specific solution suggestions from outside the immediately responsible team should be converted back into the associated problem and be subjected to all of these principles listed here.
8. Allow for and embrace idea failure
Even in ideal circumstances where we have done great research and feel that we truly understand the problem space there is a significant probability that our solution idea will still fail. The idea may fail for many reasons but in summary the majority of failure cases can be traced back to our 4 key measures of risk. Our idea may turn out to not be feasible to deliver; it may be seen as not viable by our colleagues in sales or legal; the client may see it as not usable at scale; ultimately the client may say that it does not deliver value as it does not enable them to achieve their goal(s).
The shared goal of the team tasked with crafting the solution idea - where the product designer will be playing a central role - must be to robustly chase failure as early as possible. Where a core strategy to find failures fast and early is the prototype. The team should see early failures as a positive. This is because each time it happens you are presented with a key learning opportunity.
While some commentators may view such early failure/iteration cycles in a negative light, perhaps due to the perception that the design process is taking too long, this is a short sighted view. Failing fast and early is orders magnitude less costly than fully delivering products that then fail.
9. Push for change where it is needed
Product design is at its heart the work of solving problems within any system. While the system that we are primarily interested in is our client facing product, the product designer must appreciate that our internal processes and ways of working are also systems. These internal systems we hope will mostly work in our favour, however the product designer skill set lends itself well to seeing where there is blockage or indeed failure within these systems and being able to consider a solution.
This form of internal design thinking is an important tool where the product designer will need to call on her storytelling skills to evangelise for necessary organisational change where it is discovered.
10. Always demand appropriate product telemetry
This point was made earlier but is worth repeating here given how important it is. Put simply our work to determine suitability of a solution and how we can further optimise is dependent on data. While qualitative data - for example talking to people - is important, it does not diminish the importance of having quantitive data that details how our product is used once delivered.
When it comes time to shipping our product to clients it is absolutely essential that we also ship the tools by which we can assess the use of the product. If the fact that we are deploying our software to client environments is an issue, then we just need to get more creative with a solution. For example perhaps the analytics tools need to be run periodically by the client and then physically output data to us. As perviously stated the product designer needs to get creative here and along with the rest of the team solve this problem like any other.
11. Validating in production is too late
The idea of having product use telemetry talks to the idea of increasingly our learning about our product in a client production context. This is an important form of learning but in no way should it be confused with validation.
Validation is a form of learning in which we are testing the fundamentals of a solution idea. This as mentioned above needs to be done early and absolutely before any form of technical architecture commitment. By contrast learning in production is about low level refinement and optimisation within an already validated solution context.
12. Be agnostic of upstream/downstream consumer
Lastly but by no means least it is important that the product designer operate in a way that is impartial to the downstream or upstream consumer for the part of the product in question. By this we mean given the nature of our platform that the consumer may for example be another service, an integration, an application built by a third party or perhaps a front-end of our own design. As such the product designer must avoid fixation on creating a solution that is focused on only one type of consumer. Instead a more agnostic approach is appropriate.
Product designer activity & techniques
Now that we are equipped with our core set of principles we are ready to deep dive into our five discovery phases and consider what forms of activity the product designer should be expecting to carry out. Where appropriate useful techniques are also referenced, although the reader is encouraged to experiment to discover techniques that work for them personally.
Problem definition & investigation - activity
Having arrived at a vision for the product and potentially a high level strategy, the first set of activity that the product designer will need to become involved in concerns the initial exploration of the problem space which as we have seen is perhaps best thought of as a set of scoping activity for the discovery work that will follow.
High level problem space discovery
The usage of the term discovery here might initially be confusing however at this point we are concerned with defining the shape of the problem that will form the basis of future discovery work. This activity is typically more of a product manager area of concern but it is important that the product designer, as well as the tech lead if possible, carry out this kind of scoping work together. Where the starting point will be questions like; Where has the problem come from? Has it been given to you by a client or by an internal stakeholder? What is the context in play? How pervasive is this problem within our client base? What about the wider market, are there trends or observations here that can better inform us about the nature of the problem or solution expectations?
In short at this point all we really have are questions and unless the space that the problem resides in is one that we are already very familiar with, then the product designer in conjunction with the product manager need to allow significant time to consider what are the knowns/unknowns and how we might be able to make progress with answers.
As we will return to time and again within our discovery lifecycle activities, the team need to be open to pursue any and all angles available to us to gather information that will allow us to have a view on the overall size and shape of the problem(s). If the problems have come from internal sources then what is the origin? Is there market analysis or other evidence that we have which can provide more detail? If the problem has come from a client, how significant is it? What is the presence of the problem preventing our clients from accomplishing in terms of business outcomes?
The intention here is to remain high level but to get enough signal to have a sense of the depth and breath of the problem as well as how important a solution would be for both the client and our business.
Problem space triage
It is probable that within the constraints of the product vision and high level strategy we will have a collection of problem spaces which are candidates to take further in the discovery process. Of course in order to have a view on how to do such prioritisation it is first necessary to carry out the high level discovery activity described above for each problem.
Given a collection of problem spaces and an initial set of discovery activity completed on each, it will be possible for the team to form a view on the relative priority of problems. Arguably this problem triage and prioritisation work can be considered as lower level product strategy formation. In the sense that the team are forming an initial view of what work to take forward to more intensive discovery now and what to leave till later.
In the vein of the inter-disciplinary team, discussed earlier, while problem triage may be perceived as the responsibility of the product manager, in fact a far better outcome will result when all of the roles within the team including the product designer participate in this early opinion forming. This is because each team role will bring to bear their own specialised perspective on the relative measures of risk. The product designer will be asking which problem would yield the most gains in usability. The tech lead will be assessing feasibility and cross dependancy which may dictate an order. The product manager will be thinking about where product viability can be increased the most. All three will also be thinking about which problem when solved will create the most value for the client and our business.
Existing solution exploration
Before a more complete plan can be drawn up for ongoing discovery into a problem space it is helpful to fist ask and answer the question of why the problem exists in the first place. In some circumstances this pursuit, along with the subsequent analysis step, will in fact be needed in order to effectively triage. As such it is best to consider the order of these problem definition activities as somewhat fluid.
As a product designer when faced with the question of why a problem exists one of the first areas to investigate is who/what are the actors that are faced with this problem. Then, assuming the related client business goal is not entirely new, how are these actors/consumers/users currently trying to solve this problem? The drill down questions then include:
- What systems are currently in place?
- What roles/teams are using these systems?
- What is the sort after outcome from using these systems?
- Why is this outcome currently being prevented entirely or in part?
This activity is called out as being separate from the high level discovery discussed above as even though the types of questions are overlapping, here our approach will be different. In initial discovery our enquiries will tend to focus on our colleagues in client services, sales and more general intelligence from the market. By contrast within the context of this drilling down into the cause and effect of the problem we are now wanting to focus our enquiries on the client directly.
The product designer will typically take the lead in both facilitating and carrying out these enquiries. Unless existing direct relationships are in place with the client it will be necessary to seek introductions via the relevant client services team. These introductions will likely be with a project team or vendor relations team who may be able to give you some of what you need. That said, in most cases further networking and introductions will be needed to find the persons or team that can give you the detailed answers you seek.
Technique - interviewing
When it comes to gathering understanding of a current set of circumstances and related issues it may be tempting to suggest, “Lets have a workshop, with all the right people in the room”. However in the majority of situations this is not a recommended course of action. While workshops can get you some answers fast the likelihood is that you will miss detail and not gather the range of perspectives that are typically resultant from a sequence of 1-2-1 interviews.
Having determined that interview is the course of action that is appropriate, you next need to recruit your subjects. Here 2-3 may be sufficient and certainly avoid more than 5. The key is that each interview subject should have direct experience within the area of injury. It can be helpful to ask, “are they users?”. The closer the subject is to being a user, the more valuable/accurate their information will be.
The interview itself will need preparation. A useful approach is to author a script ahead of time that includes questions as well as an introduction which spells out your objectives. It is important to use the same script for all like interviews. That said each interview should be held in an informal conversational format which will be greatly aided by keeping questions open and not feeling obliged to necessarily ask every question from the script. Interviews are most valuable when there is participation by our three key roles; product designer, product manager and tech lead. The ideal approach will be for the product designer to become the primary interviewer with the product manager and tech lead present and taking notes. So that the product designer can be fully present in the interview it is preferred if she does not take notes and instead records the audio/video for later playback.
Each interview held should become an individual page in confluence with notes and key findings. Then when the series is completed a report should be prepared, again in confluence where some analysis and conclusions are presented. To help both of these write-ups it is often helpful for the interview team to quickly de-brief after each interview. This can often yield new questions which can be taken forward to the next interview. The objective ultimately is to grow the empathy and understanding within the whole team. As such not only is the product designer primarily responsible for conducting and writing up interviews but she is also tasked with disseminated the acquired understanding to the team.
High level problem analysis
Analysis activity, in other words considering findings and drawing conclusions, is implied in all the previous steps in the sense that it is a constant activity rather than a block of activity that occurs at a specific point. That said problem analysis is called out here simply to underline the fact that having become informed about the nature of a problem space and how this may relate to other problem spaces, it is necessary to arrive at a set of conclusions in order to then shape ongoing work.
Analysis activity much be shared among all team members and in order for this to be done well it is of course essential that team members are sufficiently informed in order to make conclusions. At this stage while we may be starting to form some ideas about solution, the analysis work to be done here is more akin to arriving at a lower level product strategy.
What problems are we going to solve? What benefits/value will there be? What further research is needed in order to yield credible solution ideas? What are the likely dependancies? All these questions and more now need initial answers. That said always appreciate that new information may later present itself which will require adjustment to one or more of your initial answers. As such new information can potentially result in an adjustment to the product strategy strategy. The team need to be open and ready to do this.
Determination of target consumers
Perhaps the most important question to be answered in our early analysis is posed here separately for additional emphasis. We often simplify this question into, “who is the user?”. Though for completeness we should deconstruct this a little.
Given the nature of product it is often the case that the user, or more generically the consumer, is not a person but rather another system. In such cases it may be tempting to come to the conclusion that user experience is of less significance. The product designer needs to champion the alternative view that regardless of consumer there is always people involved. For this reason there is also always either a good or suboptimal product experience in play.
For example an API endpoint may be integrated into a downstream system by one engineering team within a client or partner. Yet further downstream the data that this API exposes is surfaced in a customer channel application by another engineering team. After this initial integration work, there is then a dev ops team which monitors the use of the API and deals with issues that arise. There is also a customer operations team which use data from this API to support customer enquiries. In a sense, all of these parties are consumers of our API.
In order to design high quality product we must decide which consumer(s) we are going to target and as such which teams within our client organisations, integration partners or within Thought Machine we need to do further research with so that we can arrive at a credible solution to the problem we are considering. Before progressing with discovery the team need to explicitly state who the target consumer is and have a good idea which teams within client orgs best represent the target consumer/user.
Design Research
In the previous section we referenced various research activities, such as 1-2-1 interviews, which are done in order to determine which problems we want to solve for which consumers. This more focused research phase is concerned with answering three primary questions as a next step:
Are our earlier conclusions about target problem space and target consumer valid?
What are the observable use cases and pain points that the target consumers have in relation to the problem space?
Are there constraints in play that will impact subsequent solution design?
Qualitative/Quantitative methods planning
There are a wide variety of research techniques that can be used to increase our knowledge. That being said given the nature of our product and the types of client relationship that we have, the likelihood is that the majority of our product learning during product research will come via qualitative methods. In other words talking directly with people and gathering knowledge through questioning.
At the start on any period of research the initial task is to consider how best to get the knowledge and understanding you seek. The quantitive space typically means looking at doing surveys or gathering data on usage, in other words analytics. As our product is not mature and many of our problem orientated projects are green field, the usefulness of these methods will be limited. This is because there will either be no usage yet or if there is data it will not be in sufficient volume to provide statistically significant signals.
As a result, as the product designer the techniques you will find yourself looking to employ are the 1-2-1 interview, which we introduced earlier, the activity observation and the workshop. Let’s briefly consider these other two:
Technique - activity observation
In many respects this is similar to the 1-2-1 interview in the sense that you have a single subject and a lead interviewer. You will also be looking to do similar preparation in advance by creating a script to shape the session. The premise of this technique is that the subject performs work tasks which are relevant to the problem that you and the team wish to understand. You are looking to gain this understanding by literally watching the subject perform these tasks. Where the product designer as the interviewer will be guiding the subject and asking exploratory questions.
Often times the observation session will make more sense after you have had an initial 1-2-1 interview with the subject. This is useful for two reasons. First it will help create context understanding around the systems in question and how the tasks of the subject fits into the wider picture of the problem space. Second an initial interview done first will create rapport and familiarity which typically will make the subject more relaxed when it comes to observation.
Subjects will often feel vulnerable when showing you how they perform their tasks, where the session may take place literally at their own workstation. For this reason the product designer must include in the script some content to make the subject feel more relaxed. Statements such as “this is not a test of you” and “we are looking to understand what helps and what hinders the work you need to do” can help to make the subject feel more at ease.
Much like with the 1-2-1 interview you would ideally perform between 2-5 separate observation sessions and have either the product manager or tech lead or both also observing. Similar write ups and report at the end will also be needed in order that the sessions will effectively contribute to both team and organisation understanding.
Technique - workshop
Earlier we discussed the fact that the workshop is generally a suboptimal technique when it comes to gathering detailed understanding of a problem space. While this is certainly true, this fact is unfortunately not widely understood by our client facing colleagues or by the clients themselves. For this reason the product designer needs to express a degree of pragmatism and know when to push back in favour of conducting 1-2-1s and when to go with the flow.
Workshops are not entirely without merit of course. Two types of situation in particular lend themselves to the workshop format. The first of these is situations in which you want to meet a team for the first time with a view to growing your network within a client. This type of session can often take place before a series of 1-2-1s where you want to get client buy in to help organise interviews as a follow-on action. Workshops of this nature often take on a sales aspect to them. Where the product designer or product manager will have prepared some materials that are used to pitch the client to participate in further research. In which you are communicating what you want to do and why it is beneficial for both parties.
The second situation where the workshop format can be very helpful is where you want to playback findings from 1-2-1 sessions for further discussion/commentary. When well prepared and presented by the product designer or product manager as “here is what we have discovered; this is what we think; we still have questions in this area”, you can get useful output. Such as findings validation/counter points, further networking referrals and increased client stakeholder engagement.
Ultimately the success of a workshop will come down to knowing exactly what the goal of the session is and vetting the participation on both sides to make sure this is achieved. It is tricky to stop the client or our colleagues from inviting one and all. However realise that the likelihood of successful workshop outcome will be dramatically reduced as you go north of 8 participants. With 5 participants being a more optimum participant number.
Participant recruitment
Clearly the success of any qualitative research study will hinge on being able to successfully recruit appropriate research subjects. This has historically been difficult to achieve within our discovery projects. However if we consider why it has been difficult we can also start to see what it will take to make this much more routine.
In 2021 the majority of our client interactions were facilitated by client services and or pre-sales. They held and cultivated the relationships with clients. Within engineering and product by contrast we had little or no client networks. Furthermore the typical client services relationship exists within the context of a project team within the client organisation, which is also typically somewhat removed from the SME or user than has the knowledge we seek. On the motivations side it is important to understand that while client services and engineering are both targeting client satisfaction, the time horizons here are not always aligned. In 2021 it was often observable that while we in engineering are looking to craft product for the long term, client-services and pre-sales are primarily concerned with a shorter time horizon.
The product designer together with the product manager need to understand and appreciate the practical implications of such an incentive mismatch. From a participant recruitment perspective this means that both these roles will need to invest significant time into growing direct networks within both our client organisations as well as our client services and pre sales teams. The latter very much being necessary in order that our colleagues become willing to show us the door, and allow us to take a copy of the key.
Primary/Secondary research studies
Having planned the studies you wish to perform and recruited your participants the clear next step is to execute the study. In our techniques considerations above we have gone into detail about the techniques that you as the product designer will perform most often. The interview, the observation and the workshop are all examples of primary research techniques in the sense that they involve participants and are carried out directly by the product designer and other team members directly.
Beyond primary research it is also worth at times considering what we call secondary research. This is simply understood as when you are able to get your hands primary research that has been performed by others parties and from where the team carries out secondary analysis on the acquired data. Perhaps you may be working with a partner who has already done some research and is sharing it with you. While we will never say no to being given more input, caution must always be taken with sourced research as you will not be fully aware of the diligence and relevance in all cases.
There are two other forms of secondary research that can be of use depending on the nature of the problem space being investigated. First there is what for simplicity we call competitor analysis. Somewhat paradoxically given the nature of our product such analysis may not always be of other core banking platform providers. Instead we just need to ask ourselves where similar problems may exist and how these problems are currently being solved. A certain degree of creativity may be required to find good proxies here. The product designer should be careful not to spend too much time on this form of study or see it as a wholly reliable input. Secondly and in a quite similar vein we have market trend analysis. This will look quite like competitor analysis but zoomed out to a more holistic/macro view.
While competitor analysis, market trends and other forms of secondary research are often interesting and can be helpful, particularly when it comes to strategic considerations such as product/market fit or opportunity sizing, the product designer needs to ensure that the significant majority of studies carried our are of the primary research type. Probably to the tune of 80% or more of research capacity.
From my experience I would be comfortable in saying that time spent on primary research is at least 8 times more valuable than time spent on secondary research.
Use case & pain point definition
The goal of this phase of discovery is to capture knowledge and form understanding about the depth and breadth of the problem space. When executed well, the product designer and the rest of the team will have significantly grown empathy for, and perhaps even camaraderie with, the consumers (users) in question. What should come next in terms of closing our this initial research stage can often be tricky for an inexperienced team to nail down.
There is no one right answer to this question of how best to sum up research activity. That said given the nature of our product I have found one approach that works well. This being to focus on detailed expression of the use cases and pain points. This is an intentionally simple framework which everyone should hopefully be able to understand.
The use cases are an accounting of low level outcomes that the consumers are seeking to achieve through the use of the part of the product in question. Here we are interested expressly in outcomes as opposed to say tasks within an existing solution. This is because when it later comes to forming solution ideas we generally do not want to be constrained by the architecture of what is current, or indeed the mental model that are expressed in the current solutions use.
Pain points by contrast typically do exist at the task level. They are the barriers, inefficiencies and suboptimal system/user behaviours which get in the way of the ideal pursuit of the desired outcomes. We want to capture and avoid them in our solution in order that what we deliver can achieve both business value (better outcome success) and user value (less pain).
The product designer needs to clearly understand that the value of such research findings to our organisation will be directly correlated to how well and how widely understood they are. Findings needs to be crafted into a simple story that colleagues can easily understand. Communication of these findings, as we have seen already with other discovery output, will need to be communicated many times to arrive at understanding. Then be communicated still more times to effectively influence the direction of the solution.
One way to achieve this is to always push for use cases to be adopted as the lens through which we both consider and express our solution ideas and prototypes that will follow next.
Solution design & architecture
Earlier we saw that the phases of discovery will not necessarily take place in a strictly linear way. In fact in most cases it makes good sense for solution design work to commence as soon as ideas become apparent. If research work is well structured then ideas will likely be coming thick and fast even from the first interview.
Use case based architecture
While jumping on ideas as soon as they become apparent can be a good thing, especially when it comes to team dynamics and motivation, we need to make sure we have done enough research diligence to avoid over emphasis on early ideas that may have little evidence to back them up. A good way to frame this risk is what is known as local maxima vs. global maxima.
Local maxima vs. global maxima
Simply put, we can often fixate on our current idea/solution and not realise that there is in fact a better idea just outside of our current vision. It is for this reason that we must make sure that if we start solution work early that we do not hold onto ideas too tightly; and at the same time also avoid investing heavily in early solutions.
We alluded to the meaning of use case based architecture in the previous section, let's now get a little more specific.
Given a researched problem space that has yielded a set of use cases/problems it is very easy for the conversation to quickly become, “the solution needs to have X capabilities”. In other words to perform a conversion of use cases/problems into a set of technical requirements. I do not think it is unfair to say that this form of reductionist thinking, a form of dehumanisation if you will, is often the path of least resistance for most engineers. A such it is the responsibility of the product designer to guard against this reductionist tendency. Instead even the most abstract consideration of solution architecture should still be expressed through the lens of our humanistic use cases and pain points.
Ideation
Within the literature on design thinking we can read about the various stages of idea formation. We typically tend to think in terms of there being a divergent phase (thinking about plausible ideas) that is followed by a convergent phase (selecting the best idea(s) to take forward). Given our product my personal view is that there are unlikely to be a large volume of competing ideas that are formed as a set which need to be narrowed down.
Instead the more likely and desirable approach will be where the team will collaboratively pursue an initial idea that would satisfy the use cases - let us call this our divergent phase. Then next to perform a series of rapid idea tests as our way to assess whether to take the idea forward - our convergent phase. Based on test outcome the team will likely form a new and better idea that is built on the foundation of the first idea, which again goes through testing. This ongoing cycle of idea refinement or progression can be thought of as honing the idea. Where after a perhaps a few rounds of divergent/convergent thinking the team will yield an idea that warrants being taken forward to a prototype.
Honing the idea
The next two sub-sections below propose the activities we should employ in the convergent phases. But what about divergent phases, how best to approach these? Here I argue that hands down the best approach is to use the brainstorming technique.
Technique - brainstorm
Given our roots in academia here at Thought Machine it is often tempting to approach solution idea formation as an individualistic pursuit. Perhaps where a single IC will form a design and then ask peers to review and comment. The product designer must be fastidious in ensuring this does not happen. While it is not impossible to get good results from such an approach, the reality is that not only will this approach take much longer but the individual mind will never be as strong, insightful or nimble as the group mind. In addition it is worth appreciating that in the solo designer circumstance the author is usually someone senior who other team members may well not feel in the position to effectively challenge the efficacy of the idea. So in short, in all cases avoid solo design and instead think groups and brainstorming.
The term brainstorming no doubt conjures a visual image in the readers mind. Perhaps they have had great or bad experiences with it in the past. Let us suspend for a moment personal metal models for this technique and rather just focus on the bare essentials. Simply put the brainstorm is an informal group meeting where we have team representation from product management, product design and tech leadership. The goal is to arrive at one of more legitimate solution idea. It is not possible or indeed desirable to be prescriptive on method. Rather the team should experiment and find what works best for them.
Approaches that I have had most success with are where participants are gathered around a whiteboard and are dynamically expressing their thought process visually. This could be in the form of a mind-map, a data resource model, our an early user journey storyboard (more on storyboards below).
The atmosphere is intended to be high energy and frenetic with an understanding that participants are unbridled by the need for ideas to be well thought out and/or correct from the outset. In such session I find that the product designer will often need to be the first to pick up the marker pen to then encourage others to follow.
Risk analysis
Given an idea that the team thinks is potentially viable - I call this an idea with legs - the team need to do some initial diligence to confirm viability. Here we can refer back to our initial discussion about accountability for the different risk dimensions. In short the product designer should take some time to consider the end-to-end usability of the solution idea - while remaining agnostic to the type of consumer as we discussed earlier. The tech lead needs to consider the feasibility side of implementation. While the product manager should be leading on viability - will the solution idea work for the rest of the business?
Of course all of the team should also be considering the value risk aspect; will the solution allow the client to satisfy their use cases and as a result generate value? Here the team will have opinions but in my experience even with the client empathy that will have been built up during a diligent research phase, the teams opinions are a lot less valuable than direct client feedback. To get this feedback we need to devise a way to test the idea.
Test the idea
It is important to stress that at this point we are not talking about the construction of a prototype or simulation of the solution. Instead we are looking for strategies that will allow us to explain to the client our thinking behind what the idea will enable, in a way that they can easily understand and as a result have an immediately expressible view on. Here we are again relying heavily on the product designer with support from the product manager in preparing the appropriate materials by which to convey the idea and solicit feedback there and then in the same session.
To be successful here we of course need to make sure we are conveying the idea to the right people - same approach as we discussed earlier in the recruitment section. Given the right audience we need to craft the message and delivery mechanism. For this I like to use the storyboard technique.
Technique - storyboard
The word storyboard is taken from the film industry. Where in order to mitigate the risk of missing a shot within a scene, an artist creates cards within which the scene is depicted shot by shot, in a manner somewhat reminiscent of a cartoon strip. Our use of the term is a little more abstract. Here product designers use their craft and storytelling abilities to describe how the target use cases would be achieved within the solution idea.
This may be a single use case for a simple idea or multiple use cases in a more involved case. The idea is to create a visual artefact that will serve as a prop to explain the sequence of actions that will be taken to achieve the desired outcome and then to have a well prepared narrative that walks the client through each step.
We need to make sure the session is long enough so that significant time is allowed for discussion and feedback. If possible get the client out of their chair and at the whiteboard to express their feedback. Done well with the right people in the room these kinds of sessions can be pure magic.
Sell the solution
The above process of brainstorming, considering risk, testing with the client may well go through a few rounds. To work well, the team will need to really buy into this idea of taking the time to refine the idea rather than jumping straight into prototyping work.
Beyond the core goal of arriving at better ideas, the above activities are also present in order to equip the team with the means to successfully sell the idea internally. Given constrained resources and competing projects it will always be the case that there is a competitive aspect to getting a project green lit. While it is possible that this step will have already occurred, often times it will be around this point where the team needs to write up the case for a project.
This step if required is typically the responsibility of the product manager. That said in the spirit of the inter-disciplinary approach that we discussed earlier, the more compelling proposals will be those where there is contribution from each of our three key roles.
In a proposal written at this stage it is critical to make clear that the exact shape of the solution is still not final. Instead the team intend to employ a variety of prototyping techniques in order to further refine the solution idea to a high level of certainty and detail. Where this prototyping work will take place before delivery of the production ready solution will commence.
Prototyping & validation
While for the most part we expect prototyping activity to occur after the project is green lit and prioritised there will again be instances where it makes sense to prototype earlier. This could be as simple as an opportunity unfolds to leverage an existing client or to work collaboratively with a new client. That said, prototyping will be most useful when the team have already tested the high level validity of an idea and now want to go to a lower level of detail.
Prototyping plan
A prototype is a mechanism by which the team can arrive at greater certainty that the proposed solution will yield a good product outcome both for the clients and for our own business. It is intended to allow the team to test one or more hypotheses with the least amount of investment possible. There are a few home truths which need to be appreciated when it comes to prototypes. A good prototype should satisfy all of the following characteristics:
1. It should be at the lowest fidelity possible to convey its purpose to the consumer;
2. It should take no longer than 1 week to construct, more optimally it would be built in a single day;
3. It is entirely throw away;
4. It has a very narrow scope rather than trying to test too many things at once.
For those readers looking to go in depth on why these points are valid then I recommend you to read the book called SPRINT by Knapp, Zeratsky & Kowitz. For our purposes here I will say that the value and utility of the prototype as a tool diminishes rapidly as time spent on it increases. Moreover the more complex the prototype is or the higher the fidelity it uses the more time it will absorb, which again will further reduce its value.
At Thought Machine we often consider our early delivery milestones as a form of prototype. The reality is that by the time our engineering colleagues have invested the man hours to reach a delivery milestone there will be significant sunk cost involved and likely a reticence to abandon work done if the prototype turns out to not be well received by the client or other stakeholders. A saying that I have which I think is particularly relevant here is:
If you aren’t learning fast you are most likely failing slowly.
Again it will be down to the product designer and product manager to continually evangelise this perspective and push for a very aggressive plan of what to prototype and what types of prototypes to employ.
Prototyping build
When approaching what kind of prototype to build it is helpful to first consider the broad types of prototype that the team may wish to make use of. There is a good exploration of this topic in the book, Inspired by Marty Cagan. To keep the references here light I propose that we might consider four classes of prototype which may feature during our discovery cycle.
Regardless of the type of prototype being used, make sure that it is used specifically to showcase the use cases that have been uncovered earlier in the discovery process.
1. Conceptual prototype
The use of conceptual here is intentional to indicate a very early prototype. It is the kind of prototype which will likely be used to convey meaning or intent to an audience in order to solicit feedback. As such this type of prototype is often used as a bridge between techniques such as storyboards and getting a green light for the project. Conceptual prototypes could also be used in the very early stages of a live project to help the team quickly answer key questions that may lead to a fork in the road for the solution.
The hallmark of this kind of prototype is that they a more likely to be used by the team to explain/show rather than used by the audience in a test scenario. If there is a visual quality then we are talking hand drawn. If there is a data quality then we are talking a spreadsheet, perhaps with some built in formulas. Think of these as another form of prop where you are helping the audience to visualise the solution, then asking them their opinion on it.
- Usually created by the product designer
2. Feasibility prototype
This type of prototype is in the realm of the engineer. Its purpose is to simulate the plausible flow of data as a way to determine if the technical architecture that is being proposed will yield a reliable, scalable, sufficiently flexible solution. Necessarily this type of prototype will involve code being written, however code here needs to be kept as lean as possible and from the outset be seen as likely entirely throw away.
A mental model that can aid understanding here is to consider this like a hackathon. In which the team have very constrained deadlines (days rather than weeks) and need to prove/disprove a very narrowly scoped solution hypothesis.
- Usually created by the tech lead
3. Usability prototype
The usability prototype is typically created a little later down the discovery path, often once baseline feasibility has been established. The goal here is to produce a fairly realistic simulation of the end experience that the solution will afford. Here it is key to realise that even if we only intend to offer APIs to the client, we will still need to test usability visually in the majority of cases. In fact most usability prototypes do not have any code sitting behind them. Instead we should think of them as wizard of oz prototypes in the sense that if there is interaction then it will be a fake simulation rather than an end-to-end service.
The other distinguishing factor of a usability prototype is that you want to be able to put it in the hands of the target user and then observe their use of it. It will not be enough to just demo it to an audience. Instead the prototype should allow the user to on their own step through what is necessary to satisfy one or more target use case. For this reason usability prototypes are typically of a high fidelity nature and generally take the form of a simulated and interactive front-end application.
- Usually created by the product designer
4. Implementation prototype
Earlier we talked about not confusing delivery milestones with prototypes that are used for validation. By this we mean that the three types of prototype above are used for validation of the solution before delivery commences. When it comes to early delivery milestones it can be useful to also think of the deliverable as a form of prototype that is used for solution optimisation rather than solution validation.
The distinction here is perhaps subtle but important. We want to be able to continue to learn and refine but we do not want to write a lot of code only to find out later that our solution is fundamentally flawed. In a sense we can think of the implementation prototype as an alpha for our software that is made available via pre-release to a vetted alpha audience (more on this later). Where we are expecting to get feedback about small change rather than significant feedback that will result in large effort code re-writes.
Usually a joint build between all members of the delivery team
While we reference implementation prototype here next to the other types of prototype, we need to appreciate that this type of prototype should not be employed as a means to validate the solution. Rather it is used in the build phase that follows as a means to fine tune the software that is delivered.
Testing the prototype
Having assembled a prototype we need to test it. This is the purpose for the prototype to exist. The method by which we test the prototype will vary depending on the type and form of the prototype however in all cases we should be thinking about how best to involve the target consumers in the evaluation.
For the conceptual prototype the method is more conversational either within the workshop or interview format. Where the product designer, with support from other team members, will be doing a combination of explaining and asking questions. With the feasibility prototype the client involvement may take the form of providing the input data by which to run a test. Where possible client data is always preferred to fabricated data as again the feasibility prototype still needs to be thought of through the lens of the use cases, rather than a purely abstract exercise.
A more involved testing approach is however called for with usability prototypes. Here the idea will follow in a similar vein to the observation sessions we covered earlier. Where instead of observing the systems currently in use you are putting the prototype in front of the user, usually via a workstation set up for this purpose. You are then asking the subject open questions about how they might carry out certain actions. With these sessions there is again a need to make the subject feel at ease. It is also important to convey to the subject that it would be most helpful if they can verbalise their thought process as they are interacting with the prototype. The product designer’s skills will be invaluable here to subtly prompt the subject where needed and extract as much feedback as possible. As with the observation sessions we are looking for 2-5 subjects that are themselves target consumers.
Evaluation & iteration
The prototype is a tool by which to increase knowledge and confidence in the validity of the solution. As such it is another example where the team will need to write up findings and arrive at conclusions. It should be noted that it may well be required to do multiple rounds of prototypes in order to address negative feedback or to otherwise change its shape based on new information. On the flip side the product designer as well as the wider team should also realise that too many rounds of prototypes are also undesirable. At some point judgment will be needed to say that enough has been done and it is time to move on to delivery.
Build evaluation & testing
Delivery or build as we may otherwise refer to it is the phase of work which follows validation of the solution. As we have covered multiple times delivery should not be used as the means by which to validate the fundamentals of the solution. However it should still be used for continuous learning and ongoing product optimisation.
Build milestones plan
When considering build milestones we are concerned with two factors. The first is the technical consideration of what is most logical to build first and where there are dependancies. The second is how quickly we can get materially useful product capability into the hands of the consumer. These two concerns can often be at odds with one another.
The product designer together with the product manager will need to advocate for an approach that will see a series of deliverable milestones that build on the one before and drip feed capabilities into our product releases. This is by far the preferred approach than say delivering nothing for an extended period and then dropping an entire solution in one release. As such the team should think carefully about how best to divide up the work in terms that makes sense for engineering and makes sense for the client.
Pre-release strategy
The purpose of each milestone is also double sided. First and foremost we want to be able to drop each new piece of capability into the hands of the target consumers in a fashion that is noncommittal from a backwards compatibility perspective. In other words we want to be able to release code and get clients to play around with it in a pre-prod type environment. While at the same time allowing us to make changes to APIs for example based on feedback. On the other side we also want to give the client capabilities they can use in production as soon a possible.
The idea of the pre-release channel speak to the former but will also ultimately pave the way for the latter. Where code is released before being fully complete so that it can be evaluated by clients using their own data. For me I would call this an alpha programme.
For an alpha programme to be successful we will need a clear process by which participating clients can become informed about the expectations we have in return for their participation. Detailed feedback must be required rather than hoped for and the timescales must be quite limited in order that learning can be sought quickly before the team is required to move on to subsequent milestone work.
Qual/quant methods for pre-prod/prod
Our qualitative methods in this phase are much the same as previous phases but will ideally centre on the observation session. What is different from previous phases is that quantitative methods will now start to become more useful. Depending on how many clients are using the alpha we can start to make use of participant surveys as well as analytics data.
Here we are mostly interested in whether the solution will still work well as usage scales. At this stage we have already arrived at high confidence that the fundamentals of the solution are legitimate but one thing that is very difficult to access during discovery is how the solution will bear up to scaled usage.
The product designer will again be active here but in fact the whole team should be continually asking themselves two questions…”how do we know that our solution is working?” and “Is usage in practice working as we intended?”.
This second question is an important one. We will often find that when it comes to usage in practice, users do not behave in the ways we expect. If you find examples of unusual or undesirable behaviour do not try to fight it. Instead lean into it and realise that this is a sign of users either moving around pain points within your solution or that a new use case is emerging that will need fresh consideration.
Restarting the cycle
While all the activity we have discussed so far will create the conditions for high probability of product success we must also realise that no discovery activity is bullet proof. Information will always be imperfect and the team will inevitably make mistakes of one sort or another. This is the balance we must always strike between time taken to achieve greater certainty and time taken to deliver a solution.
For this reason the team should realise going into discovery that the first version of any delivered software is likely to have issues with it, whether they be technical or usability related or otherwise. As such the resource plan needs to make allowances for discovery work to be ongoing and cyclical. What began as a green field problem space for us to solve for becomes rather a problem potentially of our making that will require a solution, achieved through product iteration.
Authors note
This playbook is my attempt to fully account for what good should look like both in terms of product discovery work and the discipline of product design. It is based on over a decade of experience in design roles within fin-tech and 3 1/2 years at Thought Machine. At the time of writing 13 Jan 2022 our organisation has not reached universal understanding of what product design as a discipline entails or the value it creates. I hope in creating this playbook I have been able to assist the cause of achieving this understanding.
For those readers that are product designers or interested in the discipline of product design I hope I have given you an appreciation of the vast scope of work ahead of you. Be rest assured that the competent product designer will have an outsized influence on the direction of the product and indeed the shape of the organisation. Through putting into practice the range of work we have covered I am confident that you will get drawn in and realise quickly, as I did, that if you care about product then there is no better work to be busy with.
Thanks for reading and I wish you well with your project
Hamish
