Notes on "Mythical Man-Month": "Why did the tower of Babel fall?"

Brooks uses the metaphor of the tower of Babel to again emphasize the importance of but difficulties with communication. Several times he (again) brings up problems with potential superlinear complexity owing to to the number of pairs or other groupings of workers.

"How, then, shall teams communicate with one another? In as many ways as possible: Informally... Meetings... Workbook..."

He goes into some detail talking about the workbook and processing surrounding it. It seems that online documentation has solved a lot of his problems though introducing new ones. He talks about the introduction of microfiche to avoid printing out an unwieldy number of pages, but notes the lack of ability to add notes as an important limitation. We have that problem too with online docs -- no margins to write in as an individual reader. Also, support for showing diffs in documentation is pretty uneven.

"If there are n workers on a project, there are (n^2 - n)/2 interfaces across which there may be communication, and there are potentially almost 2^n teams within which coordination must occur. The purpose of organization is to reduce the amount of communication and coordination necessary... The means by which communication is obviated are division of labor and specialization of function."

A tree structure arises because no one should have two bosses. Brooks also notes even if the reporting change forms a tree, plenty of communication outside this is necessary, forming a network.

Necessaries for any subtree:

1. a mission
2. a producer
3. a technical director or architect
4. a schedule
5. a division of labor
6. interface definitions among the parts

What Brooks calls "producer" sounds very much just like "manager" or "technical lead manager" to me while "technical director" seems to map very closely to what I think I know of as "tech lead". (In the group I'm reading this with, we had noted some of the roles he described seemed a bit unfamiliar, but here even if the names are different, the roles seem pretty close to what we have today.)

He talks about three possible arrangements between producer and director: same person, director reports to producer, or producer reports to director. He illustrates the 3rd arrangement, which I don't believe I've seen in real life, with a quote from Heinlein's "The Moon is a Harsh Mistress" where the top engineer is happy to be freed from management work.

x XOR y MOD 9 in R

I was curious what it would take to reproduce the x XOR y MOD 9 plot in R. Here's one way:

x <- 1:32
image(x, x,
    outer(x, x, FUN=function(x, y) (bitwXor(x, y) %% 9) != 0),
    col=c("white", "black"))
dev.copy(png, "~/x_xor_y_mod_9.png")
dev.off()

Looking forward to creating a GIF using the gifski package, which I learned about from gganimate, which I'm also looking forward to exploring.

Zooming out on x ^ y % 9

Having some more fun in Autoplot:

I was happy to be able to move the GIF-writing code into a library. (To keep things neat, I added a "contrib" directory in the "jython" directory within "autoplot_data". Looking at sys.path, you can see that "autoplot_data/jython" is included.)

import os.path

from contrib import gif
from org.das2.graph import DasColorBar

def xor_mod_plot(n=256, modulus=9):
  x = outerProduct(linspace(0, n - 1, n), ones(n))
  plot(gt(bitwiseXor(x, transpose(x))  % modulus, zeros(n, n)), renderType='nnSpectrogram>rebin=noInterpolate')
  dom.plots[0].setColortable(DasColorBar.Type.GRAYSCALE)

gif.plot_to_gif(xor_mod_plot, os.path.expanduser('~/autoplot/xor_mod_zoom.gif'),
   keyword_params=[dict(n=n) for n in range(4, 513)],  delay_tenths_sec=5)

The only change required for the GIF-writing code was the call to writeToBufferedImage(), which required an explicit import when used within the library:

from org import autoplot
...
        images.append(autoplot.ScriptContext.writeToBufferedImage())

Rob Pike's 6 rules of programming

@rob_pike on Twitter, 2018: "People keep quoting my '5 Rules of Programming'. Actually there were (and still are) 6 rules." He points to http://doc.cat-v.org/bell_labs/pikestyle as the best source for the rules... I will leave it to you to read the 6th rule. :-)

Animation of x ^ y % m for various m

Thanks to some example code and other generous help from Jeremy, I'm excited to have been able to create an animated GIF of x XOR y MOD m for various m in Autoplot.

import os.path

import jarray

from org.das2.graph import DasColorBar
from java import io
from java.awt import image as awtimage
from javax import imageio
import javax.imageio.metadata
import javax.imageio.stream

ARGB = imageio.ImageTypeSpecifier.createFromBufferedImageType(awtimage.BufferedImage.TYPE_INT_ARGB)

COMMON_GIF_ATTRIBUTES = {
    'disposalMethod': 'none',
    'userInputFlag': 'FALSE',
    'transparentColorFlag': 'FALSE',
    'transparentColorIndex': '0',
}

LOOP_NODE_ATTRIBUTES = {
  'applicationID': 'NETSCAPE',
  'authenticationCode': '2.0',
}

def set_node_attr(node, attributes_dict):
    for key, value in attributes_dict.items():
        node.setAttribute(key, value)

def write_gif(images, path, delay_tenths_sec=5, loop=True):
     if not images:
         return
     gif_writer = imageio.ImageIO.getImageWritersBySuffix('gif').next()
     write_param = gif_writer.getDefaultWriteParam()
     metadata = gif_writer.getDefaultImageMetadata(ARGB, write_param)
     metadata_format = metadata.getNativeMetadataFormatName()
     print metadata_format
     root = metadata.getAsTree(metadata_format)
     graphics_control_extension_node = imageio.metadata.IIOMetadataNode('GraphicControlExtension')
     root.appendChild(graphics_control_extension_node)
     graphics_control_extension_node.setAttribute('delayTime', '%d' % delay_tenths_sec)
     set_node_attr(graphics_control_extension_node, COMMON_GIF_ATTRIBUTES)
     if loop:
         app_extensions_node = imageio.metadata.IIOMetadataNode('ApplicationExtensions')
         root.appendChild(app_extensions_node)
         app_extension_node = imageio.metadata.IIOMetadataNode('ApplicationExtension')
         app_extensions_node.appendChild(app_extension_node)
         set_node_attr(app_extension_node, LOOP_NODE_ATTRIBUTES)
         app_extension_node.setUserObject(jarray.array([1, 0, 0], 'b'))
     metadata.setFromTree(metadata_format, root)
     out = imageio.stream.FileImageOutputStream(io.File(path))
     gif_writer.setOutput(out)
     gif_writer.prepareWriteSequence(None)
     for image in images:
         gif_writer.writeToSequence(imageio.IIOImage(image, None, metadata), write_param)
     gif_writer.endWriteSequence()
     out.close()
 

def plot_to_gif(plot_func, path, keyword_params=[], delay_tenths_sec=10, loop=True):
    images = []
    for params in keyword_params:
        plot_func(**params)
        images.append(writeToBufferedImage())
    write_gif(images, path, delay_tenths_sec=delay_tenths_sec, loop=loop)

def xor_mod_plot(n=256, modulus=9):
  x = outerProduct(linspace(0, n - 1, n), ones(n))
  plot(gt(bitwiseXor(x, transpose(x))  % modulus, zeros(n, n)), renderType='nnSpectrogram>rebin=noInterpolate')
  dom.plots[0].setColortable(DasColorBar.Type.GRAYSCALE)
  annotation(0, text='m = %d' % modulus, borderType='Rounded_Rectangle')

plot_to_gif(xor_mod_plot, os.path.expanduser('~/autoplot/xor_mod.gif'),
    keyword_params=[dict(modulus=m) for m in range(2, 255)],  delay_tenths_sec=50)

Chapter 4 of "The Mythical Man-Month"

Brooks: "I will contend that conceptual integrity is the most important consideration in system design." Brooks starts with a discussion of the (aesthetic?) superiority of the Reims cathedral, in which unity of design was followed, vs others where multiple architects left their mark over the centuries of work. To achieve this, Brooks suggests dividing work between architects and implementers or even using the surgical team organization discussed in the previous chapter. Thoughts:

• Reducing the number of people doing the architecture does not guarantee integrity. Even a single individual can (and will) be inconsistent if they are not careful, e.g., not being consistent in naming or ordering of input/output arguments. It seems what's needed instead are principles, including the one from Brooks here valuing conceptual integrity, and rules derived from those principles, consistently applied (ideally enforced automatically).
• The best, most harmonious set of such rules is something to be discovered. Involving more people gives more chances to discover better rules. It may be necessary to have leaders to render decisions and arrange for writing up the final product, but the flow of information is not one way.
• Brooks anticipates the objection that giving free rein to the architects may yield something impossible to implement well. It looks like he thinks part of the answer is just to hire more experienced architects... but how are they supposed to get that experience in his system? Would he count experience gained as a subordinate?
• Even if a project does end up with one architect, it seems they should be able to justify their decisions to the team, including implementers, with reference to broadly agreed upon principles and rules. The analogy with architecture makes me think of an artist who might resent having their decisions challenged. So to does the idea that integrity is achieved by having only one or two individuals make the design decisions. But we're not going for art. We're going for something that works well. (I'm imagining some parishioners sitting crushed in the rubble of a cathedral that collapsed because none of the builders dared point out a design flaw. "It hurts to have these stone blocks crushing my bones, but I really do appreciate how they're all styled the same way.")

Maybe I'm reading Brooks as advocating for a more top-down approach than he really intended. However, his discussion on "aristocracy" does seem to justify this interpretation. // Edit: I was evitably not being charitable enough. The next chapter starts with the following, emphasizing the importance of communication: "If one separates responsibility for functional specification from responsibility for building a cheap, fast product, what discipline bounds the architect's inventive enthusiasm? The fundamental answer is thoroughgoing, careful, and sympathetic communication between architect and builder."

x ^ y % 9

https://twitter.com/aemkei/status/1378106731386040322 shows a really interesting graphic generated using a simple rule: ((x ^ y) % 9) != 0. I was curious what it would take to reproduce this in Autoplot, a data browser/analysis system developed by my brother and others. Based on an example script from him, the following seems to work:

x = outerProduct(linspace(0, 255, 256), ones(256))
plot(gt(bitwiseXor(x, transpose(x))  % 9, zeros(256, 256)))

I'm pretty happy with how it turned out. I don't have time now to dig into what's going on with the image much now, but zooming in on one corner may help:

def xorModPlot(n=256, modulus=9):
  x = outerProduct(linspace(0, n - 1, n), ones(n))
  plot(gt(bitwiseXor(x, transpose(x))  % modulus, zeros(n, n)))

xorModPlot(n=32)

The 45 degree diagonal line makes sense since any integer XOR-ed with itself = 0. I was going to say that this raises another question, how much is accounted for the XOR operation alone. But, naturally, for any x != y, they differ in at least one bit and so x ^ y != 0 in that case. Notes:

• To get the script editor, remember to select "Script Panel" under the "Enable Feature" menu under "Options".
• You can find a download link for the Mac version of Autoplot on http://autoplot.org/latest/
• To get a black and white image, under the "style" panel, I selected Colortable: "grayscale" and Rebin: "noInterpolate".