Project Management, Complex Systems, Agile and Design Thinking.

Jul 26, 2025

Over the years, different methodologies to manage diverse projects have been developed and tested, with successes but also pitfalls. Different environments, industries, practices, and even regulations may turn a framework into a tool but potentially a blocker in others as well. This post is a page in my personal logbook of experiences to help me remember in the future about the process, and how the evolution of frameworks used for project management can change as one evolves professionally. Furthermore, I hope this can let the reader see a broad scope of software engineering beyond code and algorithmic structures, more focused on software development management.

Given personal origins in software industry I’ve been exposed to different methodologies to create value via software at big scale, usually in systems that are used for a large amount of stakeholders. Thereby, people building systems, likely understand the frustration that methodologies such as cascade brought to the industry. A topic well known and documented in software industry to some extend. At the same time I should recognize the utility of sensing drawbacks to design new methodologies or frameworks. But not always the transition is smooth, and many times it may be a social experiment to achieve the goal of transformation. That’s one of the lessons learned over this last 13 years in different projects. And conversely to what reader might be thinking, this is not a complaint, but a writing excercise to identify the points that let me change the mindset once new strategies are proposed.

By chance this month I decided to review a bit of literature on project management, and let me recall diversity of points to manage change and evolving with teams. A caveat in this post is the biased approach towards software industry, rather than other. The first article I read is titled Product-development practices that work: How internet companies build software. This article talks about an old dated environment where adaptability and evolution is the backbone for delivering value.

“Dealing with the software revolution requires a process that is not revolutionary but evolutionary.”

This quote resonated with me deeply, capturing the essence of how adaptability and gradual improvement are more sustainable than abrupt change in software development.

Some traits can be sensed in this article related to agile methodologies, even although it is not referenced directly. The core is cost-effective ways to rapidly adapt to environmental changes. Something we are exposed to a lot in quantitative finance, by the way. The four-stage cycle is something I personally forgot until I re-read this paper, and they are:

Identify opportunities
Analyze
Decide on
Experiment with them

All this come with teams that are empowered with operational decision rights, for sure without bypassing the hierarchical authority. Nonetheless, reader needs to be aware that this is fully linked with organizational culture for development and innovation. An attribute that can be remarked is the iterative realization of solutions that are desirable, feasible and viable. About this, I should confess I’ve had to say mea culpa, because ignoring the cultural environment has misdirected the adoption of an agile framework. Something that Microsoft did bettern when developing Internet Explorer 3, based on this article.

Direct customer feedback

Now, thinking from roles in quantitative finance I’d say some practices could be borrowed. For instance in algorithmic trading we are exposed to tons of models that evolve, and sometimes die. Evolution that is highly linked to market microestructure and why not the animal spirits described by Keyness. So, continuous feedback may be a good compass that is calibrated with expected results of stakeholders. Just think about this, imagine you take some time creating a piece of software that missed some attribute not because stakeholder’s blame, but because changing environment speed. Adaptability is the KPI for this attribute.

Divide and Conquer

In project management, this strategy refers to the structured approach for organizing tasks, managing resources, and tracking progress, ensuring that objectives are met efficiently and effectively by breaking down the project into microprojects. Each microproject is designed to deliver a subset of the functionality of the overall product.

Waterfall

This approach is one of the earliest and most traditional project management methodologies, characterized by its linear and sequential phases:

Requirements analysis
Design
Implementation
Verification
Naintenance

As linear and sequential process, each phase must be completed before the next begins, making it well-suited for projects with clearly defined requirements and minimal expected changes. Recently, waterfall approach is refered to as a methodology that contributes with structure and predictability. Nonetheless, it can be less flexible in accommodating to evolving needs or stakeholder feedback. Some details can be found in the paper Managing the Development of Large Software Systems by Dr. Winston W. Royce, which introduced the concept, and Software Engineering: A Practitioner’s Approach by Roger S. Pressman, which provides comprehensive coverage of software development methodologies.

Complex systems approach

A valuable complement to this strategy is the systems approach introduced by Ludwig von Bertalanffy in General System Theory: Foundations, Development, Applications, which emphasizes on holistic understanding of complex systems understood through the lens of open systems capable of growth and increase complexity. Thus, from a project point of view, interconnected systems rather than isolated tasks depict a more realistic modelling of systems. Therefore, a holistic mindset encourages managers and modellers to consider relationships, feedback loops, and the dynamic interactions between components, leading to more adaptive and resilient solutions. To some extend, at first glance it may look counterintuitive if pretending to understand everything upfront, but the recognition of dynamic systems allows us to talk of adaptability.

Design Thinking?

By the way, did you noticed that Ludwig von Bertalanffy was a biologist and the first brownian motion observer, Robert Brown was a botanist?. Quite interesting topic if the reader is interested on how different disciplines approach to each other, for instance as described in the book Design Thinking and other approaches - How different disciplines see, think and act by Nathan Crilly. Personally I encourage the reader to see both figure 6 and figure 7, where interaction between thinking from other disciplines are interconnected with design thinking is illustrated. Complex systems are in front of us, just that we love, as human being, to model them for forecasting and analysis purposes, among others.

Basically, the approach is commonly known as a mindset to problem-solving and innovation around human-centered design, for instance as described in this article published by Harvard Business School. It is comprised by phases that enhance the output of creational process. Additionally, this approach is complemented by Crilly, in chapter 4 of his book, argumenting design thinking is defined by an overlap of definitions. Nonehtless, his conclusion remains on that design thinking represents the process of designing.

Why is this relevant to our discussion? As part of a human-centered approach, design thinking extends beyond purely technical or abstract solutions—it bridges stakeholder needs with practical outcomes. In quantitative finance, the evolution of models over time exemplifies this process. Consider the progression from the Bachelier model to today’s sophisticated option pricing frameworks, which not only handle constant variables but also incorporate models for the parameters themselves. This reflects an ongoing evolutionary process: market participants identify new needs, and innovative solutions emerge to address real-world challenges.

Even if not formally described in quantitative manuals, this iterative approach is evident during periods of market stress, where urgency and constant feedback drive creative and effective solutions. For example, during the 2008 financial crisis or the 2020 oil price shock, quants had limited time to follow strict research protocols —necessitating rapid, creative problem-solving alongside solid research methodology. Not everything is about creativity; robust research practices remain essential when possible. If you were among the quants navigating those events, you’ll appreciate how design thinking and adaptability played a crucial role.

Agile Methodologies

I have to say that I’m not a fanatic of agile methodologies, but I recognize that it has revolutionized how teams approach software development and project management. Originally “formalized” in the Agile Manifesto in 2001, Agile emphasizes individuals and interactions, working software, customer collaboration, and responding to change. However, successfully managing Agile teams requires more than just following a framework —it demands a fundamental shift in mindset, culture, and leadership approach, and this is one of the most difficult ones depending of your team mindset flexibility.

The transition from traditional waterfall methodologies to Agile represents a significant paradigm shift. While waterfall focuses on sequential phases and comprehensive documentation, Agile prioritizes iterative development, continuous feedback, and adaptive planning. This shift has proven particularly valuable in environments where requirements frequently change and rapid delivery of working solutions is critical.

[!NOTE] I do not pretend to convert the reader into an agilist to gain the badge, rather than exposing some benefits can be taken advantage of.

Customer Collaboration Over Contract Negotiation

Agile teams prioritize direct communication with customers and stakeholders. This means regular demos, feedback sessions, and incorporating user input into the development process. Teams should establish clear channels for customer feedback and ensure that product owners maintain close relationships with end users.

Now, let’s look at how it may currently apply to quant roles. Interaction with stakeholders such as traders, portfolio managers, risk managers, back office and even regulators, makes the output from quantitative teams requiring a constant feedback and communication. For instance, an algorithmic trading strategy needs to be polished by risk management perspective to avoid unexpected losses as much as possible, align the algo-trading strategy with portfolio management goals without lossing the benefit of benefits of trading in small time intervals, and of course complying with regulations. Under this principle, communication plays the role of increasing the flow of knowledge among participants, encouraging a more collaborative environment, more than heavy documented procedures. Now, don’t get me wrong, as a model validator I’ve learned the huge benefit of having clear policy and procedure guardrails for the models. Models that many times are expressed by code (software).

Responding to Change Over Following a Plan

While planning remains important, Agile teams must be prepared to adapt when new information emerges. This requires flexible team structures, regular retrospectives, and a culture that views change as an opportunity rather than a disruption. So clear I think.

Then, adapting this to quantitative finance roles, I would say the changing environment coming from innovation in modelling, market structures, new finance product developments and even more frequent, regulation, makes us to be open to change. This change must be managed in such a way that it does not represent a blocker to internal processes or even more to innovation and development. Quite similar to the open system definition from Bertalanffy. Now, some conditions boost the effectiveness of this approach, and I’d like to refere to a nice post published by MIT Sloan Management Review in 2024, titled The four guardrails that enable agility by Nick van der Meulen. This guardrails are:

Putting purpose into action, meaning decision-making reflects the future aspirations of companies from different angles such as value propositions, core values purpose and mission. Incorporating the strategy in how change is managed ensures agile output yields benefit.
Democratize data among cross-functional teams for making operational decisions. This access needs to provide a framework that enable teams to sense the intended level of use such that experiment with relevant solutions can be performed without jeopardizing data integrity. Thereby, insights from data open the door for better understanding of stakeholders/clients.
Establishing minimum viable policies. This one might be controversial depending of the industry. Some regulations require to have clear policies that are aimed to have a well defined acting framework as safeguard of business continuity. Nonetheless, in the scope of agile, it refers to empowering teams to make operational decisions still complying with policies and standards. Many times lightweight policies help to manage exeptions more than ruling the behaviour. Balance between strong recommendations and flexibility to deviate when necessary is ideal.
Provide required resources means providing the teams with tools needed together with the environment to empower them for decision-making and product development. Conversely to traditional resource allocation, agile allocation removes the inefficiency and inability to accomodate rapid changes. This topic is related as well to Dynamic Capabilities (DC), for instance referred to by David Teece, et. al. in the article titled Dynamic capabilities and strategic management, where internal and external competences can be reconfigured in the scope of strategic management.

Individuals and Interactions Over Processes and Tools

The human element remains central to Agile success. Teams should focus on building strong relationships, fostering open communication, and creating psychological safety where team members feel comfortable sharing ideas and concerns.

Particularly some industries have some challenges due to corporative culture. But particularly in finance industry it might be complex since roles can have conflict of interets. For instance take a look on risk management (market, liquidity, for instance) departments that are deeply focused on managing unexpected losses of portfolios. On the other hand Front Office (FO) teams are full profit-aimed, in the best of the cases assessing risks. Now, let me borrowing some ideas from Litle, Brett & Shapiro shown in The strategic use of interests, rights, and power to resolve disputes. Reciprocity is a driver to manage the problem of directing communication through interests, rights and power that in many cases can bridge the gap between individual’s interaction. Nonetheless, as also mentioned by Litle et. al., some disputes can attempt to deviate from reciprocity, and likely the key is to build on top of it in negociation to strenght individual’s interaction.

Furthermore, not everything is negative. New frameworks have raised recently and some books provide some good strategies to tackle the challenges of cross-functional teams. Even if they are leadership-related, could bring an additional perspective to build relationships. I encourage reader to take a look on them, listed below.

Title	Author	Why It’s Great
Team of Teams	Gen. Stanley McChrystal	Empowering decentralized collaboration, from experience of a joint special operations task force, under uncertainty and scalability.
Leaders Eat Last	Simon Sinek	Culture and psychological safety when compared with some practices in the Marine.
Radical Candor	Kim Scott	Promotes direct but caring feedback. The narrative is quite entertaining without loosing the point.
The Culture Code	Daniel Coyle	The big idea, as it is exposed in its webpage is forming successful groups of work built according to three rules: start with safety, get vulnerable and stay vulnerable, and roadmap the story.
Trillion Dollar Coach	Eric Schmidt, Jonathan Rosenberg, Alan Eagle	Shares leadership lessons from Bill Campbell, the coach behind Silicon Valley’s top companies, focusing on trust, empathy, and building high-performing teams.
An Everyone Culture	Robert Kegan, Lisa Laskow Lahey	Encompasses questions of how organizations can thrive by fostering a deliberately developmental culture, where continuous personal and professional growth is built into the fabric of the workplace.

Common Pitfalls of Agile

While Agile methodologies offer benefits, they are not without challenges. All things aren’t good. According to Beware the Pitfalls of Agility, organizations often encounter issues when agility is pursued without clear strategic direction or alignment. Some pitfalls include:

The Pitfall of impulsiveness

Lack of strategic focus: Teams may become overly reactive, prioritizing short-term changes over long-term goals, which can dilute overall vision and impact.
Overemphasis on speed: Focusing solely on rapid delivery can compromise quality, thoroughness, and stakeholder engagement.

The Pitfall of resource fatigue

Misalignment across teams: Without coordination, Agile can lead to fragmented efforts, duplicated work, and inconsistent outcomes.
Insufficient leadership support: Agile transformations require to guide cultural change and ensure teams have the resources and clarity needed to succeed.

The balance between adaptability with strategic intent, foster cross-team collaboration, and maintain a clear sense of purpose throughout Agile initiatives avoid fall in these pitfalls.

Takeaways

As first takeaway I want to emphasize that the matter is not to choose the most recent methodology to undertake corporative environtments, characterized for being dynamic. Instead, agile frameworks could be a tool to manage the constant change, even in contexts that were not originally designed, as it is the case of quantiative finance.

Secondly, the interpersonal role is crucial for fast evolving environments, where feedback enhances the expected output and also builts on the way, not only in the end.

Third, the knowledge of all this frameworks should go through a good personal filtering process to avoid Dunning-Kruger effect, known for being a cognitive bias in which people believe they are smamrter and more capable than they are. Relevant if operational decision-making process is delegated/allowed to team members as part of an agile framework.

Fourth, in a dynamic environment the over-specialization may cause trouble such as design solution athrophy fast. This is pointed out in Product-development practices that work: How internet companies build software, section “A Team With Broad-Based Experience of Shipping Multiple Project”. Gaining relevance when projects that build a substantial experience is attributed, at least partially, to short lead times of projects in charge for team members.