What is software design?

Object-oriented technology, especially C++, seems to have brought quite a shock to the software industry. A large number of papers and books have appeared to describe how to apply this new technology. Generally speaking, the question about whether object-oriented technology is just a scam has been replaced by those about how to get benefits with minimal efforts. Object-oriented technology has existed for some time, but this explosive popularity seems a bit unusual. Why do people suddenly pay attention? People have given various explanations for this problem. In fact, there may be more than one reason. Perhaps, a breakthrough can finally be made by combining various factors, and this work is under way. However, in the latest stage of the software revolution, C++ itself seems to be a major factor. Similarly, there are probably many reasons for this question, but I want to give an answer from a slightly different angle: C++ has become popular because it makes software design easier and programming easier.

Although this explanation seems strange, it is the result of careful consideration. This article just wants to pay attention to the relationship between programming and programming. In recent 10 years, I have always felt that the whole software industry has not realized the subtle difference between making a software design and what is a real software design. As long as we see this, I think we can learn from the popular trend of C++ how to become better software engineers. This knowledge is that programming is not building software, but designing software.

A few years ago, I attended a seminar to discuss whether software development is an engineering discipline. Although I don't remember the result of the discussion, I remember how it made me realize that the software industry has made some wrong comparisons with hardware engineering and neglected some absolutely correct ones. In fact, I don't think we are software engineers because we don't realize what real software design is. Now, I am more convinced of this.

The ultimate goal of any engineering activity is certain types of documents. When the design work is completed, the design documents are handed over to the manufacturing team. Team and design team are completely different groups, and their skills are completely different from those of design team. If the design document correctly describes a complete design, then the manufacturing team can start manufacturing products. In fact, they can start building many physical objects of the product without any further intervention from the designer. According to my understanding, after reviewing the life cycle of software development, I come to a conclusion that the only software document that truly meets the engineering design standards is the source code list.

There have been many arguments about this point of view, and there are enough pros and cons to write countless papers. This paper assumes that the final source code is the real software design, and then carefully studies some results brought by this assumption. I may not be able to prove that this view is correct, but I want to prove that it does explain some observed facts in the software industry, including the popularity of C++.

When the code is regarded as the result of software design, one result completely overrides all other results. It is very important and obvious, and because of this, it is completely a blind spot for most software organizations. As a result, the construction cost of software is very low. It does not have expensive qualifications at all; It's cheap, almost free. If the source code is software design, then the actual software construction is completed by compilers and connectors. We often refer to the process of compiling and connecting a complete software system as "one-time construction". The main investment of software building equipment is very small-all that is needed is a computer, an editor, a compiler and a connector. Once you have a build environment, the actual software construction only takes a little time. It may take a long time to compile a 50000-line C++ program, but how long will it take to build a hardware system with the same design complexity as 50000-line C++ program?

Another result of treating source code as software design is that software design is relatively easy to create, at least in the mechanical sense. It usually takes only a few days to write (that is, design) a representative software module (50 to 100 lines of code) (completely debugging it is another topic, which will be discussed in detail later). I really want to ask whether other disciplines can produce designs as complex as software in such a short time, but first we must figure out how to measure and compare complexity. However, it is obvious that software design can quickly become very large.

Assuming that software design is relatively easy to create and inherently inexpensive to build, it is not surprising to find that software design is usually very large and complex. This seems obvious, but the importance of the problem is often overlooked. Projects in schools usually have thousands of lines of code. Software products with 10 000 lines of code (design) are also discarded by their designers. We have long since stopped paying attention to simple software. The design of typical commercial software consists of thousands of lines of code. Many software designs reach millions of lines of code. In addition, software design is almost always evolving. Although the current design may only have a few thousand lines of code, you may actually have to write many times of code during the product life cycle.

Although some hardware designs seem as complicated as software designs, please pay attention to two facts about modern hardware. First of all, complex hardware engineering results may not always be error-free. At this point, there is no criterion that we believe in like software. When most microprocessors are sold, there will be some logical errors: bridge collapse, dam breach, plane crash and the recall of thousands of cars and other consumer goods-all of which are fresh in our memory, and all of which are the result of design errors. Secondly, complex hardware design has a corresponding complex and expensive construction stage. Therefore, the capacity required to manufacture such a system limits the number of companies that can really produce complex hardware designs. For software, there is no such restriction. At present, there are hundreds of software organizations and thousands of very complex software systems, and the number and complexity are increasing every day. This means that it is impossible for the software industry to find a solution to its own problems by imitating hardware developers. If there is anything in common, it is that when CAD and CAM can help hardware designers create more and more complex designs, hardware engineering will become more and more similar to software development.

Designing software is an activity of managing complexity. Complexity exists in the software design itself, in the company's software organization and in the whole software industry. Software design and system design are very similar. It can span many technologies and often involves many branches of disciplines. Software specifications are usually not fixed and often change rapidly, which often happens in the process of software design. Similarly, software development teams are often unstable and often change during the design process. In many ways, software is more like a complex social or organic system than hardware. All these make software design a difficult and error-prone process. Although all these are not creative ideas, today, nearly 30 years after the software engineering revolution began, compared with other engineering industries, software development still looks like an untrained skill.

It is generally believed that when real engineers finish a design, no matter how complicated it is, they are very confident that the design can work. They are also very confident that they can use recognized technologies to build designs. In order to do this, hardware engineers spend a lot of time verifying and improving their designs. For example, consider a bridge design. Before such a design is actually built, engineers will conduct structural analysis-they will build computer models and make simulations, and they will build scale models and test them in wind tunnels or other ways. In short, before the construction, the designer will use all the methods he can think of to prove that the design is correct. For the design of new passenger aircraft, the situation is more serious; It is necessary to build a prototype with the same size as the original one, and flight tests must be carried out to verify various expectations in the design.

For most people, software is obviously not as strict as hardware design. However, if we regard the source code as a design, we will find that software engineers have actually done a lot of verification and improvement on their design. Software engineers call it testing and debugging, not engineering. Most people don't regard testing and debugging as a real "project"-certainly not in the software industry. The reason for this view is more because the software industry refuses to regard code as design than any actual engineering differences. In fact, test models, prototypes and circuit test boards have become recognized components of other engineering disciplines. The reason why software designers don't or don't use more formal methods to verify their designs is because of the simple economic law of software construction cycle.

The first revelation: just building a design and testing it is cheaper and simpler than doing anything else. We don't care how many builds are built-the time cost of these builds is almost zero, and if we discard the build, the resources it uses can be completely reused. Please note that testing is not only to make the current design correct, it is also a part of improving the design process. Hardware engineers of complex systems often build models (or, at least, they use computer graphics to visualize designs). This gives them a "feeling" about the design, but it is impossible to get this feeling just by checking the design. It is neither possible nor necessary for software to build such a model. We just make the product itself. Even if formal software verification can be done automatically like a compiler, we will still go through the build/test cycle. So formal verification has never had much practical significance for the software industry.

This is the reality of the current software development process. More and more people and organizations are creating more complex software designs. These designs will be written in some programming languages and then verified and improved through the build/test cycle. This process is error-prone and not particularly strict. Quite a few software developers don't want to believe that this is how the process works, which makes the problem more complicated.

At present, most software processes try to divide different stages of software design into different categories. The top-level design can't start coding until it is frozen. Testing and debugging are only to eliminate construction errors. The programmer is in the middle. They are construction workers in the software industry. Many people think that if we can make programmers stop hacking and build according to the design given to them (and make fewer mistakes in the process), then software development can become mature and become a real engineering discipline. However, as long as the process ignores engineering and economic facts, this will not happen.

For example, any modern industry can't stand the rework rate exceeding 100% in its manufacturing process. If a construction worker often fails to do well the first time, he will soon lose his job. But in the software industry, even the smallest piece of code may be modified or completely rewritten during testing and debugging. In a creative process (such as design), we realize that this improvement is not part of the manufacturing process. No one expects engineers to create a perfect design the first time. Even if she does, she still has to go through the improvement process to prove that it is perfect.

Even if we don't learn anything from Japanese management methods, we should know that blaming workers for mistakes in the process is not conducive to improving productivity. We should not always force software development to follow incorrect process models. On the contrary, we need to improve the process to help rather than hinder the production of better software. This is the touchstone of software engineering. Engineering is about how you realize the process, not about whether you need a CAD system to produce the final design document.

There is an overwhelming problem about software development, that is, everything is part of the design process. Coding is design, testing and debugging are part of design, and we usually think that design is still part of design. Although the construction cost of software is very low, the design cost is incredibly high. Software is very complicated, and there are many different aspects of design content and design considerations. The problem is that all the different aspects are interrelated (just like hardware engineering). We hope that top-level designers can ignore the details of module algorithm design. Similarly, we hope that programmers don't have to consider top-level design issues when designing internal algorithms of modules. Unfortunately, problems at one design level can invade other levels. For the success of the whole software system, choosing an algorithm for a specific module may be as important as any higher-level design problem. There is no level of importance in different aspects of software design. A wrong design in the lowest module can be as deadly as a mistake in the highest module. Software design must be complete and correct in all aspects, otherwise, all software based on this design is wrong.

In order to manage complexity, the software is designed in layers. When a programmer is considering the detailed design of a module, there may be hundreds of other modules and thousands of details, which he cannot consider at the same time. For example, in software design, there are some important aspects that do not completely belong to the category of data structures and algorithms. Ideally, programmers should not consider other aspects of design when designing code.

However, design doesn't work like this, and the reason is beginning to become clear. Software design is incomplete before writing and testing. Testing is a basic part of the design verification and improvement process. The design of high-rise structure is not a complete software design; It's just a detailed structural framework. Our ability to strictly verify high-level design is very limited. In the end, the influence of detailed design on high-level design is at least as great as other factors (or this influence should be allowed). Improving all aspects of design is a process that should run through the whole design cycle. If any aspect of the design is frozen out of the improvement process, it is not surprising that the final design will be bad or even unable to work.

It would be great if advanced software design could become a more rigorous engineering process, but the real situation of software system is not rigorous. Software is complex, it depends on too many other things. Perhaps, some hardware doesn't work as the designer thinks, or a library routine has a limitation that is not explained in the document. Every software project will encounter such problems sooner or later. These kinds of problems will be found during the test (if we do the test well), because there is no way to find them early. When they are discovered, they will force the design to be modified. If we are lucky, the changes to the design are partial. Often changes will affect some important parts of the whole software design (Murphy's Law). When a part of the affected design cannot be changed for some reason, then in order to adapt to this influence, other parts of the design must be destroyed. This usually leads to what managers think is "random coding", but this is the reality of software development.

For example, in a project I recently participated in, I found the time dependence between the internal structure of module A and another module B. Unfortunately, the internal structure of module A is hidden behind an abstract body, which does not allow the calls to module B to be combined into its correct call sequence in any way. When we found the problem, of course, we missed the opportunity to change the abstract body of A. As expected, what happened was to apply more and more complex "corrections" to A's internal design. When we haven't finished installing the 1 version, it is generally felt that the design is going downhill. Every new revision may destroy some old ones. This is a formal software development project. Finally, my colleagues and I decided to change the design, but in order to get the approval of the management, we had to work overtime voluntarily and without pay.

In any large-scale software project, such problems are inevitable. Although people have used various methods to prevent it from appearing, they still ignore some important details. This is the difference between technology and engineering. If experience can lead us in the right direction, it is technology. If experience will only lead us into unknown areas, then we must use the method we used at the beginning and make it better through a controlled improvement process, which is engineering.

Let's have a look. As a small part of it, all programmers know that writing software design documents after coding rather than before will produce more accurate documents. The reason is obvious. The final design represented by code is the only improvement in the build/test cycle. In this cycle, the possibility that the initial design remains unchanged is inversely proportional to the number of modules and programmers in the project. Will soon become worthless.

In software engineering, we need excellent design at all levels. We especially need excellent top-level design. The better the initial design, the easier the detailed design will be. Designers should use anything that helps. Structure diagram, Butch diagram, state table, PDL, etc. -If it helps, use it. However, we must remember that these tools and symbols are not software design. Finally, we must create a real software design, and it is done in programming language. Therefore, when we start designing, we should not be afraid of coding. We must be willing to improve them when necessary.

So far, no design symbol can be applied to both top-level design and detailed design. The design will eventually be expressed as code written in programming language. This means that the top-level design symbols must be converted into the target programming language before the detailed design begins. This conversion step takes time and introduces errors. Programmers often review the requirements and redesign the top-level design, and then code according to their own actual situation, instead of converting from a symbol that may not be fully mapped with the selected programming language. This is also part of the reality of software development.

Perhaps, it will be better if the designer writes the initial code himself instead of letting others modify the language-independent design later. What we need is a unified symbol suitable for all levels of design. In other words, we need a programming language that is also suitable for capturing high-level design concepts. C++ can just meet this requirement. C++ is a programming language suitable for real projects and a very expressive software design language. C++ allows us to directly express high-level information about designing components. In this way, the design can be carried out more easily and the design can be improved more easily in the future. Because it has a more powerful type checking mechanism, it also helps to detect errors in design. This leads to a more robust design, which is actually a better engineering design.

Finally, software design must be expressed in programming language, and then verified and improved through the construction/testing cycle. Any other ideas beyond this are completely useless. Please consider what software development tools and technologies are popular. Structured programming was considered as a creative technology at that time. Pascal made it popular and became popular himself. Object-oriented design is a new and popular technology, and C++ is its core. Now, please consider those things that are invalid. Case tools, are they popular? Yes; Is it universal? No. What about the structure diagram? The situation is the same. Similarly, there are Warner-oltu, Butch Map, Object Map and everything you can think of. Each one has its own advantages, only one fundamental weakness-it is not a real software design. In fact, the only universally recognized symbol of software design is PDL. What does it look like?

This shows that in the common subconscious of the software industry, instinctively knowing the improvement of programming technology, especially the programming language used in actual development, is of overwhelming importance compared with anything else in the software industry. It also shows that programmers care about design. Software developers will use more expressive programming languages when they appear.

Also, consider how the software development process has changed. Once upon a time, we used the waterfall process. Now, we are talking about spiral development and rapid prototyping. Although these technologies are often considered as "eliminating risks" and "shortening the delivery time of products", they are actually just to start coding earlier in the software life cycle. This is a good thing. This allows the build/test cycle to start verifying and improving the design earlier. This also means that top software designers are likely to carry out detailed design.

As mentioned above, engineering is more about how to realize the process than what the final product looks like. In the software industry, we are close to the standards of engineers, but we need some cognitive changes. Programming and build/test cycle are the center of engineering software process. We need to manage them in this way. The economic law of the build/test cycle, coupled with the fact that software systems can represent almost anything, makes it completely impossible for us to find a universal method to verify software design. We can improve this process, but we can't break away from it.

The last point: the goal of any engineering design project is some document products. Obviously, the documents actually designed are the most important, but they are not the only documents to be produced. Eventually, someone will use this software. Similarly, the system is likely to need subsequent modification and enhancement. This means that auxiliary documents are as important to software projects as hardware projects. Although the user manual, installation guide and other documents that are not directly related to the design process are temporarily ignored, there are still two important requirements that need to be solved by using auxiliary design documents.

The first purpose of auxiliary documents is to capture important information from the problem space, which cannot be directly used for design. Software design needs to create some software concepts to model the concepts in the problem space. This process requires us to understand the concepts in the problem space. Usually, this understanding will contain some information that cannot be directly modeled in software space, but this information will still help designers determine what basic concepts are and how to best model them. This information should be recorded somewhere in case you want to change the model later.

The second important requirement of auxiliary documents is to record some aspects of the design, which are difficult to extract directly from the design itself. They can be high-level content or low-level content. For many of them, graphics are the best way to describe them. This makes them difficult to include in the code as comments. This does not mean that graphical software design symbols should be used instead of programming languages. This is no different from the graphic design document that supplements the hardware subject with some text descriptions.

Never forget that it is the source code, not the auxiliary documents, that determines the real appearance of the actual design. Ideally, you can use software tools to post-process the source code and generate auxiliary documents. We may expect too much from this. Next, programmers (or technical writers) can use some tools to extract some specific information from the source code, and then they can record this information in other ways. There is no doubt that it is difficult to update such documents manually. This is another reason to support programming languages that require more expressiveness. Similarly, this is one reason to support keeping this auxiliary document to a minimum and formalizing it as late as possible in the project. Similarly, we can use some good tools; Otherwise, we have to turn to pencils, paper and blackboards.

Summarized as follows:

The actual software runs in the computer. It is a sequence of 0 and 1 stored in some magnetic medium. It is not a program written in C++ (or any other programming language).

A program list is a document representing software design. In fact, it is compilers and connectors that build the software design.

It is incredible how cheap it is to build an actual software design, and it always becomes cheaper as the speed of computers increases.

The cost of designing actual software is unbelievable, because the complexity of the software is unbelievable, and almost all steps of the software project are part of the design process.

Programming is a design activity-a good software design process recognizes this, and when coding makes sense, it will not hesitate to code.

Coding shows its meaning more frequently than we think. Usually, the process of expressing design with code will expose some omissions and additional design requirements. The sooner this happens, the better the design.

Because software is very cheap to build, formal engineering verification methods are of little use in actual software development. Just building a design and testing it is simpler and cheaper than trying to prove it.

Testing and debugging are design activities-for software, they are equivalent to design verification and improvement processes in other engineering disciplines. A good software design process recognizes this and will not try to reduce these steps.

There are other design activities-called high-level design, module design, structural design, architecture design and so on. A good software design process recognizes this and carefully includes these steps.

All design activities are interactive. When different design steps show the necessity, a good software design process recognizes this and allows design changes, sometimes even fundamental changes.

Many different software design symbols may be useful-they can be used as auxiliary documents and tools to help simplify the design process. They are not software designs.

Software development is still a craft, not an engineering discipline. The main reason is that it is not rigorous in the key process of verifying and improving the design.

Finally, the real progress of software development depends on the progress of programming technology, which in turn means the progress of programming language. C++ is such an improvement. It has gained explosive popularity because it is a mainstream programming language that directly supports better software design.

C++ has taken a step in the right direction, but it needs more progress.