W. L. Cleveland, Ph.D.
March 27, 2008
The Cellbot project is aimed at the development of an autonomous system for the automatic analysis and manipulation of cultured cells at the single-cell level. The Cellbot utilizes an automatic microscope to acquire low power images with transmitted light that are analyzed by pattern-recognition algorithms (Support Vector Machines (SVMs)) to recognize and localize viable cells in physiological cultures. Higher magnification images of the identified cells obtained with fluorescence techniques can be analyzed by other SVM algorithms to identify patterns of biological significance. Decision-making algorithms can utilize these pattern recognition events to control the robot, making further observations, perturbing cells at the single-cell level by microinjection, or micromanipulating individual cells at relevant time points for assay of gene expression and other parameters. The Cellbot project brings together multiple technologies and skills that have been acquired over a long period of time as a result of basic research interests. The significance and nature of the Cellbot project is best understood in relation to this background
I began my career in science as an undergraduate, working as an Instrument Maker in the laboratory of Sven Hartmann in the Department of Physics at Columbia University. I was under supervision of Norman Kurnit and joined the laboratory just after Norman had made the first observations of photon echoes.
As part of a long tradition, it was common for graduate students and postdoctoral fellows in the Columbia Physics Department to build instruments that were beyond the commercially available state of the art in order to study important basic questions. It was in this “instrument maker” tradition of physics that I began my scientific training and it is in this tradition that all of the technology development in my career has been done. Here I would note that technology development in the instrument maker tradition of physics is very different from the engineering tradition. Instrument fabrication by physicists is often done by individuals whose intellectual roots are in both the technology domain and in the basic science application domain. Moreover, it is the basic science interest that is the driving force of the technology development. In the engineering tradition, it is common for an engineer to have intellectual roots only in the technology domain and to know little about what is referred to as the “problem domain.” Consequently, individuals developing technology in the physics tradition tend to have a more interdisciplinary focus than those working in the engineering tradition. There is also a tighter coordination of technology development and basic research in the physics tradition.
In the late 1960s, the Columbia Physics Department was a very vibrant and well funded place that offered a phenomenal opportunity to learn the technical skills of experimental physics. It possessed a well-equipped machine shop staffed by skilled machinists. However, the need for formal drawings and a single draftsman in the department could often be a bottleneck. Part of my job was to bypass the bottleneck. Having learned machine shop practice in high school and being able to work with only a sketch, I was asked to make components that were used in neodymium and ruby lasers and optical delay lines. I also gained experience in optics, electronics and cryogenics.
As a graduate student in the Chemistry Department at Rutgers, my instrument maker skills proved to be useful for a molecular spectroscopy thesis project. I made a variety of devices, including a nitrogen laser, a dye laser, and high voltage power supplies for conventional flash lamps. Experience with chemical synthesis and chemical purification, including the preparation of ultrapure compounds by zone refining, as well as various spectroscopic techniques (fluorescence, phosphorescence, IR, Raman, all at 4.2 oK) was also gained.
After completing my training in molecular spectroscopy and quantum chemistry, I became a postdoctoral fellow in the Erlanger laboratory in the Department of Microbiology at Columbia. In the Erlanger laboratory, I began my career as a basic immunologist, focusing on immune regulation, especially idiotype regulation, and on fundamental aspects of T-cell antigen recognition. After becoming immersed in these very different areas, I did not expect that I would ever again need the instrument-maker skills I had acquired in my student days. However, after establishing my own laboratory at Roosevelt Hospital, I developed a strategy for immunotherapy of chronic lymphocytic leukemia that was based on killer T-cell recognition of the processed fragments of the immunoglobulin v-regions expressed by malignant B-cell clones. It was in this project that the need emerged for robotic instrumentation that was not commercially available.
In my NIH R01-funded project entitled: “An Idiopeptide Vaccine for Chronic Lymphocytic Leukemia”, we succeeded in establishing immortalized malignant B-cell lines as well as long-term cultures of non-immortal malignant B-cells. To carry out meaningful studies of the immortalized cells, cloned lines were needed. This proved to be very difficult and ultimately required combinations of cytokines and non-transformed feeder cells that were derived from the patient and were in short supply. To deal with these difficult logistics, we constructed a manual micromanipulator that proved very useful.
Our interest in micromanipulator technology was further stimulated by our experience with the cultures of nonimmortalized malignant B-cells, which were unintentionally established during an attempt to culture dendritic cells. To our surprise, these cultures, which contained both dendritic cells and nondividing malignant B-cells, survived for as long as 5 months, raising the possibility that our culture conditions were a mimic of the in vivo conditions that lead to the central enigma of chronic lymphocytic leukemia (CLL): extremely high numbers of circulating malignant B-cells that are paradoxically in G0. It should be appreciated that CLL malignant B-cells die rapidly in ordinary culture media supplemented with serum, suggesting that our special conditions for coculture with dendritic cells were critically important. Unfortunately, for a definitive conclusion (and for publication), we needed to measure gene expression in single cells isolated at multiple time points over periods of months from large numbers of very small cultures. Our inability to address these issues with manual techniques became a major motivation to develop a robotic and more extended version of our manual cell manipulation device. However, this agenda was not actively pursued until we became aware of the IMAT program of the National Cancer Institute, which funds technology development projects without requiring the hypothesis-driven research that is essential for successful R01 applications.
We succeeded in getting IMAT funding for a project that we entitled: “Robotic Preparation of cDNA from Single Cells.” The initial goal of this project, which we now refer to as “The Cellbot Project”, was deliberately restricted to the initial development of a semiautomatic system that would require a human operator. This was done to avoid the fatal criticisms of “overambitious” and “lacking in focus” from grant reviewers. However, one reviewer of our proposal clearly appreciated our basic vision for the project and encouraged the development of a fully automatic system in which the human operator is replaced by machine vision algorithms.
Our first step toward the development of an autonomous system involved the use of an artificial neural network (multilayer perceptron) that was trained to recognized cells in simple brightfield images. In the course of this work, we discovered that artificial neural networks can recognize unstained living cells in microscope images with a capability that rivals that of a human observer. This discovery was subsequently extended to support vector machines which are much more readily optimized than artificial neural networks. This achievement generated the opportunity to build a highly autonomous microscope and cell manipulator that was much more powerful than the system we initially envisioned.
The autonomous system with the capabilities described above can be compared in its sophistication with the driverless vehicles that drove themselves along a 132 mile long course through the Mojave Desert in the $2,000,000 DARPA Grand Challenge competition. In both cases, algorithms must simulate human operators performing highly sophisticated recognition and control tasks.
A unique feature of our system is its capability to extract single-cell-level information from small cultures with minimal perturbation. This capability complements the flow cytometer which performs best as a high-speed device for manipulation of large numbers of cells. The need for relatively large numbers of cells in flow cytometry means that small cultures must be destroyed or significantly perturbed to harvest sufficient cells for flow-cytometric manipulation or analysis. Our technology completely bypasses this problem and provides a solution for a class of cell analysis/manipulation problems that is currently not addressed by any established technology.
As noted above, this technology development project was initiated in response to our own need to manipulate small cultures of leukemia cells. However, in our grant proposals we emphasized that the Cellbot was in effect a robotic tool box that would have general applicability in cell biology. We also noted it usefulness in stem cell research. This latter application has recently been made much more important as a result of the recent discovery by Shinya Yamanaka that induced pluripotent stem cells (iPS) can be derived from adult skin cells. This discovery promises to generate a major movement in biological research that represents an ideal opportunity for our technology to make a significant contribution. As noted in the February 1, 2008 issue of Science: “To validate iPS, scientists must make huge [numbers of iPS lines] from many different people and compare them in a battery of tests with human ES [embryonic stem] cells.” This “industrial scale” effort will inevitably require automated methods. We believe that our “autonomous tool box for cell biology” is well-positioned to make an important contribution to this dramatic breakthrough.
The application of the Cellbot to stem cell research is also of importance to our current research agenda, which has evolved in recent years to include a focus on molecular mechanisms in major psychiatric disorders. We hope to test hypotheses for these disorders using in vitro cultures of iPS-derived neurons and glial cells from specific patients for whom genetic polymorphisms are also known.
Currently we have well-developed cell-recognition algorithms as well as a fully motorized microscope and cell manipulator. The next step in the development of our system will involve the use of established techniques to couple our cell-recognition algorithms with our hardware to generate an autonomous system that can automatically identify cells in a culture and transfer individual cells from one location to another. Achievement of this milestone will provide a functional test system that will facilitate the development of the sophisticated capabilities described above.
Long X, Cleveland WL, Yao YL, "Multiclass Detection of Cells in Multicontrast Composite Images,"Computers in Biology and Medicine 2010, 40: 168-178.
Long X, Cleveland WL and Yao YL, “Automatic Detection of Unstained Viable Cells in Brightfield Images Using A Support Vector Machine with an Improved Training Procedure,” Computers in Biology and Medicine 2006, 36(4): 339-62.
Long X, Cleveland WL and Yao YL, “A New Preprocessing Approach for Cell Recognition,” IEEE Transactions on Information Technology in Biomedicine 2005, 9(3): 407-412.
Long X, Cleveland WL and Yao YL, “Effective Automatic Recognition of Cultured Cells in Brightfield Images Using Fischer’s Linear Discriminant Preprocessing,” Image and Computing Vision, 23, 2005, pp.1203-1213.
Long X, Cleveland WL, and Yao YL, “Effective Automatic Recognition of Cultured Cells in Brightfield Images Using Fischer’s Linear Discriminant Preprocessing,” Proceedings of IMECE04: 2004 ASME International Mechanical Engineering Congress, November 13-19, Anaheim, California USA, 2004.
Long, X, Cleveland, WL, Yao, YL. Methods and systems for identifying and localizing objects based on features of the objects that are mapped to a vector. 7,958,063