signals and slots

C makes U hang yourself;
C++ even gives U a rope object to hang;
C# scourges U using a rope wrapper!

rarely talk about programming unless it comes to some new idea or something fun. However, in this article, I’m going to summarize a part of experiences in my career, about important concepts that I’ve learned and used: signals and slots. First appear and remain as fundamental concepts in the Qt library (“Qt” pronounced “cute” – which is so cute as a cross-platform application framework), the concepts has been adapted and developed into some other libraries (such as boost) and are being proposed as next standards for C++ language.

Please note that we’re talking C/C++ here, although signals & slots has been brought to many other languages (Java, Python, Ruby, JavaScript, etc.) Every coin has two sides and C/C++ is no exception, it has its own advantages and disadvantages (depend on types of applications). But the thing I like about C/C++ is that its half – civilized nature made it a strong language: the balance between abstraction and comprehensiveness has always been considered as core factor.

Noone want to build general applications using Assembly, and noone want to write lengthy code just for some simple features. To adhere the doctrine, strong – abstraction concepts must be introduced to capture grammar of the language, and simple, clear syntax that would make the code concise and comprehensive.

I came to signals & slots as solution to problems I’ve met when leading the CMS project, a DVRs managing software (client & server) with complicated GUI: a few screens, each screen can have a dozen tabs, each tabs can have hundreds of controls. Events on one control can affect other controls on other tab or screen, events can be user inputs as well as various signals come from dozens of (managed) network client machines.

The overwhelmingly – complicated communications surpass every MPM (message passing method), then signals & slots appeared as a saviour solution. To be simple, signals & slots is a very efficient mean of interaction between objects within a process space. I personally think it would be the next metaphor extension added to C/C++ language. Structure of this post would be as follow:

1. Introduction to signals & slots, from raw to abstract concepts.

2. ISO C++ compliant and cross – compiler, cross – flatform problems (would discuss about delegate and how function pointer has evolved from C to C++).

3. Different implementations of signals & slots: Qt, boost, sigslot, libsigc++

1.   INTRODUCTION

Flow of code is no more than a series of function call, and to perform asynchronous function call, traditional C programming use callback. A callback is a pointer to a function, so if you want a processing function to notify you about some event, you pass a pointer to another function (the callback) to the processing function. Callbacks have two fundamental flaws: firstly, they are not type-safe, we can never be certain that the processing function will call the callback with the correct arguments. Secondly, the callback is strongly coupled to the processing function since the processing function must know which callback to call. In C, to define function pointer, you would write something like this:

float (*my_func_ptr)(int, char *);
typedef float (*MyFuncPtrType)(int, char *);
MyFuncPtrType my_func_ptr;

When move to C++, things are a bit more complicated (note the weird syntax ->* to call a member function through pointer).

float (SomeClass::*my_memfunc_ptr)(int, char *);
my_memfunc_ptr = &SomeClass::some_member_func;
SomeClass *x = new SomeClass;
(x->*my_memfunc_ptr)(6, "Another Arbitrary Parameter");

Though a very strong concept in C, function pointer add little to the language C++. The only two uses of member function pointer in C++ are:

  • Demonstrate the tricky syntax of the language.

  • Help in implementation of delegate.

Now, imagine slots as invisible communication channels, every object can register to listen to some channels by providing delegates (pointer to a member function). The delegate would be invoked when slot is “filled” with a signal. The signals and slots mechanism is type safe: signature of a signal must match signature of the receiving slot. Signals and slots are loosely coupled: a class which emits a signal neither knows nor cares which slots receive the signal. To summarize the benifits:

  • Syntactically neater: signals & slots make for very readable code: you define the signals, you define the slots, then you wire them up.

  • Inherently robust: when either end of a signal/slot connection is destroyed, the connection is automatically removed. It is therefore impossible to emit a signal and have it arrive at a class that has already been deleted. Best of all, you don’t need to write any cleanup code.

  • Easier code reuse: since the thing that has to type-agree is the signal and the slot, not the whole class interface, it is much easier to plug together disparate classes that had never initially been intended to work in that way.

2.   DELEGATE

To summarize section 1: pointer is the root of all evils. Function pointer is the physical of callback method traditionally used in C programming. When evolved to C++, the C++ standard committee failed to address how to define function pointer to a single C++ object. With common-sense knowledge, we know that it take two elements to represent the thing: first is the address of the object itself (the implicit pointer this), second is pointer to the object’s member functions. The undefined land leads to different implementations across various compilers.

Contrary to common belief, the actual physical storage of pointer differs between various compilers, and even differs between types of pointer, e.g: pointer to a single – inheritance object, an multiple – inheritance one or virtual inheritance. The table below shows how many bytes it take to store pointers on different compilers (just make use of the operator sizeof). Then to calculate the actual member function’s address, each compiler uses one kind or another of offset:

struct BorlandMFP
{
        CODEPTR m_func_address;
        int delta, vindex;
};
if (vindex==0) adjustedthis = this + delta;
else adjustedthis = *(this + vindex –1) + delta;
C++
compiler
void*
pointer
function
pointer
single
inheritance
multiple
inheritance
virtual
inheritance
MS VC++ 4 4 4 8 12
GNU G++ 4 4 8 8 8
Borland C++ 4 4 12 12 12

So, delegate is just the new name for pointer to (an object’s) member function. The hard part really is that: since each compiler stores different type of pointers differently, there’s need for a way to store delegate in a universal manner: cross compilers and cross flatforms. Various techniques have been proposed by excellent programmers. Please refer to the articles below for specific techniques. I personally would prefer the method of Quynh Nguyen, an engineer at Global Cybersoft (Vietnam) Ltd.

3.   IMPLEMENTATIONS

To make those obscured concepts above clearer, I would go into some detailed implementations of signals & slots with the emphasize on the sigslot library, for this is really a compact lib: everything is contained in just one header file. Suppose you have a server that would receive some signal from network (tcp socket), then parse the received data and trigger some processing (display on GUI, store to disk…). We would define the “network signal” as follow (please note any number of parameters can be defined, we use an int and a char* here just for the example):

// the temmplate ‘signal2’ denote for
// a signal with 2 parameters

signal2<int, char*>m_netSig;

Then, at the place you want to process that event, e.g some class CDisplayForm, you would need to provide a function to process the signal:

DisplayForm::OnNetSig (int cmd, char* data)
{
  // the delegate would be invoked
  // when the signal is recieved

}

Please note that multiple objects can register to a same signal slot (and vice versa), e.g some class CRecordThread would also need to log that event to database. Then the listening objects would just register with the signal slot using a connect function:

// we pass the function pointer
// here to be called later

m_netSig.connect (this, &CDisplayForm::OnNetSig);

When the network signal arrive, e.g in some class CAsyncSocket, we would fire the signal and get it populated to all connected objects:

// emit the signal with it’s data
m_netSig.emit (NETSIG_DATA_RECEIVE, xmlStr);

Then the CDisplayForm and CRecordThread object would both receive the signal. Please note that when either end of the communication channel go out of scope, the connection is automatically removed, you don’t have to write any cleanup code. Just define the signal, the slot and wire them up! Any unrelated classes can be wired up in this type-safe manner without any pre-defined relation within them.

Speaking as an ex-hardware designer, I’d like software components to have pins around their edges, like ICs, so they can be ‘wired up’, but otherwise should be well behaved… I don’t want to know how that pin is wired internally – I just want to know that it will work when I send a signal into it.
(Sarah Thompson – sigslot library)

ipix fish-eye camera


the dome camera and it’s Nikon fish-eye lens

Image above: Ipix’s raw format (spherical image). Click to try the Ipix static image viewer in a separate window, which helps to view a single Ipix’s frame, wrapping from raw format to normal rectangular format (not the video).

his is among the most interesting things I’ve ever worked on: Ipix’s CommandView™: fish-eye cameras which provide a 360° x 180° field of vision. This camera and it’s technology were very special and advanced at the time, about 3, 4 years ago, it were so new that the manufacturer could not survive the small market share and went bankruptcy. However, the technology has now revired at http://www.ipix.com. The camera came with some totally new concepts, patents and technologies. Some information on Ipix:

Ceiling or wall-mounted, with thick hard steel cover, the camera is designed to operate indoor or outdoor, and can withstand a temperature range from -40°C to 60°C. It costs about $2100 at the current time, and about $5000 when it first appeared. CommandView™ Dome is an IP camera, having a small Linux computer inside, once plugged onto LAN or WAN, the camera turns into a video server and can stream video in many protocols: http, ftp, rtp, or multicast. There’re 3 ways to control the camera: ssh to the Linux machine, access to it’s web-server or programatically use the XML-RPC interface. The product has a very matured and solid development flatform.

CommandView™ Dome is a mega-pixel camera (2 MB, 3 MB). The unique feature of this product is that it provides a 360° x 180° field of view with no moving parts. See the image beside (on the left – click to view a mega-pixel version), every details in the room is captured on a parabolic sphere (called wrapped image). To have a normal view, a region on the sphere is dewrapped (by software) into usual rectangular plane.

Since a whole 360° space is captured, PTZ (pan / tilt / zoom) is just a matter of dewrapping, there’s no need to motorize the camera for different sighting fields. This special camera has broadened the definition of ‘panorama’ and has many applications in security surveillance as well as photography. I myself found it very interesting to apply the concepts and softwares to demonstration and show-case purposes (search for ‘ipix’, the name has gone far beyond just a type of camera).

file system encryption


The pc-link reader, could be in many forms: usb, serial, pcmcia, pinpad…

An Axalto (previously Schlumberger’s, now Gemalto) smart card, this could be an access – control card, a bank debit / credit card, health – care card, social – identification card… Some smart card types are actually tiny computers – they have an Operating System, and Java virtual machine inside. Note: many of the smart card vendors are OEM – ed from SCM Microsystems, a Germany – based company. We’d better choose hardware and firmware directly from SCM when work on open – source projects.

ou have important data on your laptop, and one day the laptop is stolen. The best you can expect is that you loose all those valuable information. The worse… who know what it can be!? Since having a computer (hardware) in touch means having access to every data stored on it. How can you protect yourself from that situation?

The solution is encrypting your storage and keep your key safe from others’ access. You can encrypt your home directory or a whole data partition. (Some even encrypts /boot, /swap and /(root) – full disk encryption – so that their activities leave no traces). With Linux, every tools is at your hand, so let start making your life easier. Below is some guide lines on how to do it on Debian.

1.   Install eCryptfs: a Linux native and POSIX – compliant enterprise – class stacked cryptographic file system.

$ aptget install ecryptfsutils
$ modprobe ecryptfs
# you may have to patch your kernel in order for
# ecryptfsd (the key management daemon of eCryptfs)
# to work. in that case, you’d better grab and
# build eCryptfs from the latest source.

2.   Setup eCryptfs:

# simple setup, passphrase entered from stdin,
# eCryptfs allows overlay mounting: mount point
# and the actual storage can be the same.

$ mountt ecryptfs /encrypted/storage /mount/point
# you can now see your new fs using the command df,
# test it, go to your mount point and input some
# data, watch the output at the encrypted storage.
$ cd /mount/point
$ echo “Hello World” > hi.txt
$ cd /encrypted/storage
$ less hi.txt
# umount /mount/point and the file system
# is not accessible anymore

HOW TO PROTECT YOUR KEY?

Every security solution comes to a very dead-end that there’s must be a MASTER KEY as the main entrance for the whole system. And we have several ways to protect the master key:

+   the tricky way: store the key somewhere on the disk, e.g: at the first sector of the volume image (offset some bytes to a position only you know) so that the key is hard to find and can not be accidentally deleted. More information about the trick is here.

+   the explicit way: store on an external storage such as usb, the system could only be functional once the usb is plugged in.

+   the “smart” way: smart card is the best way to store key. Since you can write public / private key to a smart card but can only read public key back, no – one can read the private key, including you. Smart card is designed to do encryption / decryption jobs: just request the card to encrypt / decrypt some data using a specified key, and the card sends back the required output.

In an organization, the private key is stored on smart card and given to the “person in charge” without the fear of loosing the key. For personal use, I think the second solution (using usb) is quite enough.

1.   Setup smart card:

# install pcscd, the daemon to talk with
# card reader install OpenSC, the open
# source smart card project

$ aptget install pcscd
$ aptget install opensc
# in my case, I have an Axalto reader which is not
# supported by pcscd. knowing that Axalto’s
# hardwares are OEM-ed from SCM, I just download
# the firmware from SCM, flash the reader and it’s
# then recognized as a SCM SCR 331 device. You can
# play around the smart card using opensc-tool,
# you can erase, init the card’s file system,
# create some PIN, generate some keys using
# pkcs#15-init, encrypt/decrypt can be
# done using pkcs#15-crypt

2.   Working with keys:

Public and private keys are fundamental concepts of Digital Signature. In short, multiply two big prime numbers – two private keys – you have a semi-prime number – the public key. Certificate is wrapper of the public key to work with a CA (Certificate Authority), for the public to check if a public key really belongs to an organization.

# generate public and private keys in RSA 1024
# bit pem format extract the public key from this
# keys pair, encrypt/decrypt with the keys

$ openssl genrsaout private.pem 1024
$ openssl rsain private.pemout public.pemoutform PEMpubout
$ openssl rsautlencryptinkey public.pempubinin file.txtout file.ssl
$ openssl rsautldecryptinkey private.pemin file.sslout decrypted.txt
# create the smart card’s file system,
# then create a PIN

$ pkcs15initcreatepkcs15
$ pkcs15initstorepinauthid 01label “mycom”
# copy private key to smart card, to decrypt,
# we need to specify key’s usage

$ pkcs15initstoreprivatekey private.pemauthid 01id 45format pemkeyusage sign,decipher

3.   Setup the whole thing:

With your smart card properly setup (a public/private key pair has been stored), we can use the following script to: 1. create a temporary volatile file system 2. use smart card to decrypt the password and store to that file system (these files would simply disappear when the machine power down) 3. using the decrypted password to mount with eCryptfs.

# 1. create a volatile fs to temporarily store the
# decrypted passfile

$ mountt tmpfso size=10M,nr_inodes=10k,mode=0700 /tmp_fs
# 2. decrypt the password file from
# /root/passwd to /tmp_fs/passwd

$ pkcs15cryptpkcs1decipherk 45i /root/passwdo /tmp_fs/passwd
# 3. now mount our encrypted
# directories with eCryptfs

$ mountt ecryptfs /encrypted/storage /mount/pointo key=passphrase:passfile= /tmp_fs/passwd, cipher=aes, ecryptfs_key_bytes=16, passthrough=1, verbosity=0

Done! Choose a strong encryption algorithm (e.g AES 256 – bit), and in most cases, you and your data would be safe until they have quantum computer running. Be sure to keep your key secret or your efforts would be of no help. For maximum security, choose a good smart card model, increase your keys’ size, revise all procedures of your software system…

(You’re pretty safe right now, so why there’s an “in most cases” in my last paragraph? The truth of being secured is actually more complex than that, as a quite-simple technique like this could be applied to steal password of a disk-encryption system. This bases on the fact that data in DRAM is not faded right away after loosing power, it still can last for some seconds (or even minutes), and by cooling the DRAM using an air duster (to slow down the fading speed), the DRAM module could than be copied and scanned for password (assuming that password has been entered and kept in memory). Nothing is really safe, however, hardware problem like this should have a hardware solution (like this AES 256-bit encryption Fujitsu hard drives).

CMS 2.0 released

Some typical UI features:

Data & input validation using XPath

Custom-drawn combobox (with check-box item)

Custom-drawn grid with tens of thousand cells

Simple vector object editing

Custom-draw tree with loading effect

he CMS project, in which I’m the team leader, has come to it’s second phase, with major features completed. Although I felt tired with the on-going work, it’s my most beloved product. CMS (short for Central Management System) serves as a gateway through which clients manage the DVR Servers (Digital Video Recorder). The product demonstrates many technologies & programming techniques, just to name a few:

  • IOCP (IO completion port): the multi-threaded IOCP server has proved to be very efficient, hundreds of concurrent connections won’t rise CPU usage to higher than 5%. However, the connections are stateless, it’s best that we would add a state-machine to better control the traffic.

  • TCP + XML communication: we devise a draft protocol like XML-RPC or SOAP, which bases on TCP rather than HTTP.

  • XML data storage: all network command & data is presented in XML. With version-numbered XML, we hope to solve the backward compatibility problem ever existed with so many releases of our main DVR products.

  • XML processing is made easy thank to the wonderful TinyXML and TinyXPath libraries. I must emphasize that XPath makes XML processing much more feasible, it greatly reduces the number of LOC (line of code) and procedural complexity.

  • A sophisticated client MFC GUI demonstrates various GUI programming techniques, almost every tricks with MFC GUI has been gathered in this product: heavy custom-draw controls, flicker-freed rendering, advanced input validation…

  • Some borrowed concepts from other flatforms: the elegant “signal & slot” are added to handle complicated interaction between GUI’s elements (in addition to MFC’s message queue mechanism).

I have some lessons learned after the project:

  • It’s a MUST that you have a team with members of the same professional level. This would make sure the members would easily understand each others and share a common culture.

  • C/C++ is still a MUST for real time programming tasks (it’s “half-civilized nature” makes it a strong language). And the use of some matured flat-form: boost, wxWidget… would greatly help the developing process.

  • It’s only by the desire for creation & creativity that programmer would offer an excellent product. The culture of a developing team is somewhat like honey-bees, who collect pollen little by little to make honey and wax.

I’ve been involved in many projects, some is more interesting, some is more difficult than this one. But this is the first one that I’d made, everything from A to Z. It’s now time to leave I3 but I must say this is the project that I loved and two years at I3 was a special time in my career! Looking at the screenshots below, you would see that CMS is very rich in it’s designs about information presentation.

a Japan that can fail

ave you ever asked yourself why 80 percent of software projects fails, why there’s so much argues and debates in our IT community? If you have the courage to look deeply into yourself, to see our nature and the nature of the IT field, you would figure out things quite easily. Pity that most people can not do this. I read the article below for quite some times. It’s idea is quite straight-forward, however, I made a Vietnamese translation for one who still can not get it.

Bạn thân của bạn muốn nói với bạn một số điều mà anh ta biết có thể làm bạn đau. Cả hai cùng đến một câu lạc bộ riêng tư nào đó, nơi không ai khác nghe thấy được. Anh ta không muốn nói một chút nào, nhưng là một người bạn tốt anh ta phải nói. Sau một vài ly, câu chuyện sau đây được kể.

CÂU CHUYỆN MỘT NƯỚC NHẬT CÓ THỂ THẤT BẠI

Đối diện những đe dọa đau đớn của sự hủy diệt, các nền văn hóa thường từ chối tự nhận chân chính mình. Họ rút lui vào một xó xỉnh nào đó, bày tỏ thái độ bàng quan, và tìm kiếm an ủi trong những huy hoàng của dĩ vãng. – Fouad Ajami – The New Republic 8/91.

Sau chiến tranh thế giới lần thứ 2, nước Mỹ thống trị kinh tế toàn cầu, hơn bất kỳ thế lực nào khác đã (thậm chí là sẽ) làm được. Dù cho các doanh nghiệp Mỹ tin rằng điều này sẽ kéo dài mãi mãi, đó thực sự là một trạng thái không tự nhiên, các doanh nghiệp ngày càng trở nên nặng nề và lười biếng.

Lấy thí dụ điển hình là công nghiệp ôtô. Vào lúc người Nhật bắt đầu xâm nhập và cạnh tranh trên trị trường này ở Mỹ, thì các doanh nghiệp Mỹ đã trở nên quá thiếu hiệu quả và ngại thay đổi. Điều này khiến người Nhật nhanh chóng chiếm lĩnh những phân khúc lớn của thị trường. Các công ty Mỹ đã quên kỹ năng cạnh tranh trong một thời gian.

Tuy vậy, người Nhật phát triển được nền kinh tế siêu cường của họ, phần lớn dựa vào các ưu điểm của văn hóa Nhật Bản hơn là dựa vào thực tế công nghiệp sản xuất ôtô Mỹ lúc đó đã quá cồng kềnh và kém hiệu quả. Và người Nhật cũng bắt đầu tin rằng sự thống trị của họ trong một số ngành công nghiệp là tự nhiên và sẽ kéo dài mãi.

Còn nguy hiểm hơn, họ cho rằng, áp dụng các phương pháp, công thức Nhật Bản vào các ngành công nghiệp khác sẽ mang đến thành công tương tự. Cuộc sống sẽ là quá dễ dàng nếu chỉ là như thế. Hầu hết doanh nghiệp Nhật Bản còn tin chắc rằng cứ kiên trì phương pháp của mình, sớm hay muộn họ sẽ thành công.

Đây là khởi đầu cho câu chuyện nước Nhật có thể thất bại. Lấy ví dụ trong các sản phẩm mang tính nghệ thuật (phim, đĩa nhạc, phần mềm…) nước Nhật không đóng một vai trò đáng kể nào. Các công ty Nhật có thể mua lại các công ty Mỹ, và thuê người Mỹ làm ra các sản phẩm Mỹ, để bán trên toàn thế giới, nhưng thực sự số lượng sản phẩm xuất khẩu này rất ít. Nhật Bản không thực sự là một tay chơi lớn trên loại thị trường này.

Vấn đề không phải là người Nhật đã không thử. Hầu hết đều nhận ra là làm phần mềm có lãi hơn là phần cứng. Trong thời đại kinh tế thông tin, lãi sinh ra từ các băng đĩa VCR thì ít, mà từ các phim ảnh ghi trên đó thì nhiều.

Vấn đề cũng không phải là họ thiếu những con người tài năng. Một đất nước sản sinh ra được những Yukio Mishima và Akira Kurosawa là đã có đủ những nguyên liệu thô cần thiết để sản sinh những tài năng xuất chúng của thế giới. Nhưng liệu những người như thế có làm việc cho các công ty Nhật không, hay các công ty Nhật có thuê những người đó hay không? Ngày nay, không có chỗ cho Soichiro Honda, người chưa từng tốt nghiệp trung học giữa những nhân sự tất cả đều tốt nghiệp đại học Tokyo và cũng xin vào một vị trí thử việc.

Vấn đề thực sự ở đây là người Nhật xem phần mềm như một ngành kỹ nghệ. Hầu hết các công ty lớn của Mỹ cũng mắc phải sai lầm này, đó là lý do chỉ một số ít công ty lớn của Mỹ thành công về phần mềm (trừ trường hợp của Microsoft, những công ty thành công là những công ty vừa và nhỏ về nhân sự).

Phát triển phần mềm có hai đặc trưng cơ bản khiến nó có tính nghệ thuật hơn là khoa học. Thứ nhất là quá trình thiết kế phần mềm. Phần mềm sẽ tương tác với người dùng như thế nào, và làm sao làm được điều đó? Quá trình thiết kế vừa tiếp diễn, vừa tập trung tại mọi công đoạn. Ngày hôm qua là thiết kế trên nền DOS, hôm nay đã là Windows. Từng ngày một trong quá trình phát triển đều có những quyết định mang tính thiết kế, dù nhỏ, dù to. Bạn không thiết kế phần mềm để rồi sau đó viết một cuốn sách về nó.

Cũng giống như làm phim, bạn không thiết kế đến từng chi tiết rồi sau đó tiến hành. Đạo diễn, người quan trọng nhất của một cuốn phim không thực sự làm việc cho đến khi bắt đầu bấm máy. Đến lúc đó, từng diễn viên mang phần đóng góp trên vai của mình vào cuốn phim, và quá trình thiết kế tương tác và tiếp tục suốt quá trình làm phim. Chỉ một diễn viên, hay đạo diễn tồi có thể làm hỏng bộ phim, cũng như cách một lập trình viên kém làm hỏng toàn dự án.

Đặc tính thứ hai của phát triển phần mềm là: quá trình giải quyết vấn đề cần đến những cải tiến liên tục. Mỗi ngày lại có một lập trình viên nào đó có một cách tốt hơn để giải quyết vấn đề. Nếu bạn cứ phụ thuộc vào những phương pháp cũ, phần mềm của bạn sẽ lạc hậu rất nhanh chóng. Một đối tượng hiển thị phông chữ cố định (fixed font) sẽ không được dùng (cho dù bạn test nó nhiều như thế nào) nếu mọi người đều muốn dùng phông chữ tỉ lệ (proportional font). Máy tính cũ hơn 5 năm sẽ bị vất đi ở Mỹ cùng với các phần mềm đi kèm với nó.

Và điều quan trọng là: cải tiến và sáng tạo không xảy ra trên dây chuyền sản xuất. Nó không xảy ra vào một giờ cố định trong ngày. Nó không xảy ra khi lệnh và sự vâng lời được xem trọng hơn là ham muốn tạo ra một sản phẩm nổi bật.

WEB BẰNG THÉP KHÔNG RỈ

Khi tự tin thoái lui, ngờ vực ngự trị. Khi ngại khó, “ti mít” chiếm thượng phong, thì trí tượng tượng, sự chủ động mà mọi công ty cần có để giữ vững tốc độ tăng trưởng bị thách thức bởi những xoi mói, phê bình ác ý. Đây là nguyên nhân khiến cho các công ty trở nên tê liệt và thụ động, ngay cả khi đã có được đồng thuận về vấn đề, về những hành động cần có, và ngay cả về những thay đổi cần thiết. Khi đó, một đề nghị thay đổi sẽ bị kẹt cứng trong một mạng lưới (web) bằng thép không rỉ, cơ hội tồn tại của nó trước những kẻ bàng quan thờ ơ thật quá xa vời. – John Walker – The Final Days, Autodesk, Inc. Information Letter #14

Các công ty Nhật Bản cần phải tự thay đổi. Câu hỏi không phải là phát triền phần mềm theo cách nào?. Không có câu trả lời tường minh nào cho điều đó, tuy vậy, chỉ cần có một thay đổi toàn diện và triệt để trong cách quản lý công ty. Tôi không muốn lặp lại những gì đã viết trước đây (bài Software Design, tháng 11, 1991). Điều quan trọng là vất bỏ phương pháp công xưởng, nhà máy ra khỏi quá trình phát triển phần mềm. Thay vào đó, bạn tạo các nhóm nhỏ độc lập, mỗi nhóm có cách của riêng mình. Bạn sẽ chọn lọc và thay thế các cách cũ bằng những cách tốt hơn. Và mỗi nhóm được tự do đi theo cách của riêng mình, miễn là nó hiệu quả.

Những quy luật cũ, dù tường minh hay ngấm ngầm đều phải được bỏ đi. Cá nhân là quan trọng hơn nhóm, nhóm là quan trọng hơn cả công ty. Mỗi cá nhân đều được khuyến khích SÁNG TẠO. Khi mỗi cá nhân đi lệch một chút trên hướng đi của họ, ý tưởng mới sẽ nãy sinh. Và mỗi người giải quyết một mảnh của vấn đề, công việc được giải quyết theo một cách mới và tốt hơn.

Các công ty Nhật vẫn có thể là những tay chơi lớn vì dù xu hướng cá nhân trong văn hóa Mỹ rất mạnh, điều đó lại không phổ biến trong lĩnh vực phần mềm này. Tôi đã gặp một vài lập trình viên Nhật Bản, khi đi phỏng vấn ở Nhật và ở MSKK (chi nhánh Nhật của Microsoft). Hầu hết những người này, nếu xét riêng, cũng giỏi như những nhân sự giỏi nhất ở Mỹ. Họ đáng được cho những cơ hội được làm việc như những lập trình viên, vì với những người đó, sáng tạo là một phần thưởng cá nhân quan trọng hơn nhiều so với phải làm việc trong một môi trường nhà máy.

Cũng như các công ty ôtô Nhật đã nhận ra rằng nhân sự Mỹ thật sự tốt và đã mở các nhà máy ở Mỹ, các công ty phần mềm Mỹ cũng sẽ nhận ra những tài năng tiềm ẩn trong những nhân sự Nhật Bản và mở những trung tâm phát triển phần mềm ở đây.

Tuy vậy, ngồi không mà không làm gì tiềm ẩn những hiểm họa lớn. Nhật Bản sẽ vẫn còn phụ thuộc Mỹ ở những công nghệ phần cứng cốt lõi nhất. Giải quyết những vấn đề này sẽ phải vi phạm một vài nguyên tắc cơ bản nhất của văn hóa Nhật. Với một số người, cái chết là lựa chọn tốt hơn so với việc đánh mất danh dự.

Nghệ thuật cũng phy lý tính như là âm nhạc. Nó điên khùng trong vẻ đẹp riêng của nó – George Jean Nathan (1882-1958)

PHẦN KẾT

Kết thúc những lời này, người bạn tốt nhìn vào chính bạn. Anh ta hy vọng là những lời nói dữ dội này không ảnh hưởng đến tình bạn giữa hai người. Điều quan trọng là anh ta hy vọng bạn sẽ nhìn nhận sự thật này trước khi quá muộn. Vì thường thì khi vẫn đề đã trở nên rõ ràng không chối cái được nữa, thì cũng là lúc đã quá muộn để sửa chữa.

 

Your best friend needs to tell you something that he knows will pain you greatly. The two of you go to your private club and you sit where no one else can hear you. While your friend’s greatest wish is to not talk about this, he knows that as a true friend he must tell you what he believes. So over drinks he tells you of a problem that he thinks affects your entire future. This is what he tells you.

A JAPAN THAT CAN FAIL

On pain of extinction, cultures often stubbornly refuse to look into themselves. They retreat into the nooks and crannies of their perceived history, offer up the standard evasions, fall back on the consolations they know. – Fouad Ajami in The New Republic 8/91.

Shortly after World War II, the United States dominated the world economy more than any other country ever has (or probably ever will). Although businessmen in America came to believe that it would continue indefinitely, this was an unnatural state of events and many of our industries became fat and lazy.

The American automobile industry is a prime example. By the time Japanese companies started competing in America, the American companies were so inefficient and resistant to change that the Japanese companies quickly captured a large segment of the market. The American companies had forgotten how to compete.

Japan took its economic accomplishments as a sign of its superior culture rather than a sign of fiercely competitive companies taking on some rigidly structured, inefficient American car manufacturers. As with America after WW II, Japan is starting to believe that its dominance in certain industries is a natural state of events that will continue indefinitely.

Even more dangerous, many Japanese believe that by applying the Japanese method to any industry, they will be able to dominate it too. Life would be so easy if it were so. Unfortunately, when the Japanese method is applied to industries like software it fails. Even more unfortunate, most Japanese companies convince themselves that if they continue to use the same old methods, sooner or later they will succeed.

This is the Japan that can fail. In the world markets for artistic products (movies, records, software, etc.), Japan isn’t even a player. Yes, Japanese companies can buy American companies that hire American workers in America to create American products that sell all over the world; including Japan. However, I doubt that Japan’s exports for these types of products are greater than that of any single European country. This is a market that America totally and unequivocally dominates and Japan isn’t even a player.

This isn’t for lack of trying. Most Japanese companies realize that its more profitable to write software than to build hardware. In the information age, economic power won’t come from building the VCR. It comes from producing the movies on the tapes.

The problem isn’t for lack of qualified people. A country that produced a Yukio Mishima and Akira Kurosawa has the raw talent to produce some of the greatest artistic efforts in the world. However, does anyone think either of these people could ever have worked for a modern Japanese company? Or that a Japanese company would hire them? Where is there room for a Soichiro Honda who never graduated from high school among all of the graduates from Tokyo University competing for entry level jobs?

The major problem is that Japanese companies treat software development as engineering. (Most large American companies make this same mistake which is why you see very few large companies even in America successfully producing software.) Unfortunately, object-oriented programming gives the factory approach a whole new way to become even less efficient. Software companies can now produce object after object and document each one so that someone else can find it and use it.

The problem is that people are now spending a lot of their time documenting their objects and searching for other objects. And when you do find an object, it usually turns out that it’s not quite what you wanted anyway. And finally, even when objects give you what you want, your core problems haven’t been eliminated.

Software development has two basic characteristics that make it an art. The first is the design process. How is the software going to interact with the user and what will it do? This design process is ongoing and occurs at every stage of development. Two years ago programs were written for DOS. Now they are written for Windows, which requires a totally different design. Virtually every day of programming includes design decisions; some big, some small. You do not design and then write a book.

You do not design to the last detail and then shoot a movie. The director of a movie is considered to be the single most important contributor to a movie yet they are generally not involved until the shooting arts. Each actor & actress brings their own interpretation of their character to the movie, creating an ongoing design of the movie. In the same way, just as a bad actor or director can ruin an otherwise great movie, one bad programmer can ruin an otherwise good program.

The second basic characteristic is that problem solving requires constant innovation. Every day some developer somewhere comes up with a better way to solve a problem. If you continue to depend on existing methods, pretty soon your software is so outdated that no one will use it. An object that only displays a fixed font is of no use, no matter how well tested, in a world that now requires proportional fonts. Computers that are five years old are being thrown out in America as useless and the same should be happening with almost all code written that long ago.

Constant innovation and creativity don’t happen on an assembly line. They don’t happen during certain hours of the day. They don’t happen when concensus and order are more important than creating an exceptional product.

THE STAINLESS-STEEL WEB

When self-confidence fails, caution rules. When timidity and unrestrained risk-aversion gain the upper hand, the kind of imaginative and bold initiatives that companies must make in order to sustain their rapid growth are forced to run a gauntlet of analysis and criticism that no suggestion, no venture not already proven successful, has a chance of surviving. This is the source of paralysis, why companies fail to act even when there is, within the company, a broad concensus that problems exist, action is needed, and even agreement on the nature of the changes required. Even then, each specific recommendation finds itself stuck in a stainless-steel web; the chances of it surviving all the individual sign-offs needed to be implemented are remote. – John Walker in The Final Days, Autodesk, Inc. Information Letter #14.

The world is changing today. Russia has become one of the greatest capitalists of all with its offer to sell the Kurile Islands back. Who would ever have thought that Russia wouldn’t care if Japan gets the islands back as long as they get as much money as possible for them. For the last quarter, Microsoft’s profits were almost as great as those of IBM; a company with 40 times the number of employees. What worked yesterday is no guarantee for tomorrow.

Japanese companies need to change too. The answer isn’t to find a Japanese way to develop software, its to use they way that works – steal it from America. There is no great secret to how to effectively develop software, however, it does require a complete change in how Japanese companies manage software development.

I don’t want to repeat what I have written earlier (see the article in the November 91 issue of Software Design) but essentially the factory needs to be thrown out of the software development process. To replace it, you create small independent teams that will each develop in their own way. You will replace one method with many methods. And each group is totally free to try any way they want – as long as it works.

The old rules, both explicit and implicit must be cast off. The stainless-steel web must be destroyed. Concensus and deliberation no longer dominate. The individual becomes more important than the group. The group becomes more important than the company. Each individual is encouraged to CREATE. And as each developer goes off in their own direction something wonderful starts to happen – new ideas come in. As people solve each part of a problem, it’s done in new and better ways.

Japanese companies can be a major player in the world software market. While the American culture has an advantage with its emphasis on individuality, it’s not an overwhelming one. I met a number of Japanese programmers, both when I was interviewed for a job managing developers in Japan and developers from MSKK (Microsoft’s Japanese subsidiary). Most of these are people who, if let loose, could be the equal of the best in America. Moreover, these people should be given the opportunity to be developers. For most people, creating is much more rewarding personally than working in a factory environment.

Just as Japanese car companies discovered that there is nothing wrong with the American worker and opened plants in America, American software companies will discover the talent hidden in Japanese developers and open software development centers in Japan.

The risks of not acting are great. Japan will be dependant upon American companies for the brains in their hardware. Solving this problem requires violating some basic tenets of Japanese culture. To some, death may be preferable to dishonor.

Great art is as irrational as great music. It is mad with its own loveliness. – George Jean Nathan (1882-1958)

THE END

After he has finished your friend looks at you. He is hoping that speaking this bluntly did not hurt your friendship. More importantly, he hopes that you will see the truth before it’s too late. Because by the time a problem is inescapably obvious, it’s usually too late to rectify it.