Wrong Research Results

Don’t believe everything you read. It’s a truism no rational person would disagree with. But how does it apply to published scientific research?
When teaching my graduate level lithography class at the University of Texas, I often caution my students about excessive faith in published research. “Half of what is published in the lithography literature is wrong,” I would often say. I have no data to support this claim, but after reading thousands of published papers over the last 23 years I think that number is in the ball park. I’ve recently read a published scientific paper (yes, the irony is thick here) that puts some scientific backing to the claim that most published research is wrong. John P. A. Ioannidis’s paper “Why Most Published Research Findings Are False” is fascinating – a must read for anyone involved in research. Published in the peer-reviewed open access journal PLoS Medicine (August, 2005), the paper is slanted towards medical studies, but the principles apply to all of science. When is data sufficiently strong to justify the statement that it supports either the acceptance or rejection of a proposed hypothesis? Most of the time, he claims, the data supports neither statement.
The conclusion? Science definitely moves forward, but a healthy dose of skepticism is justified.

New web item – my recent lithography papers

I’ve just added a new page to my web site under Published Works called Chris’s Recent Papers. The goal is to put the last few papers I’ve written here for easy access (as links to the papers or PDFs to download). If I keep writting these kinds of papers, the list should change frequently. Here is what is up there now:

What’s So Hard About Lithography?
Accuracy, speed, new physical phenomena: The future of litho simulation
Methods for Benchmarking Photolithography Simulators: Part IV
Fast lithography simulation under focus variations for OPC and layout optimizations
From Data to Decisions

SPIE Microlithography Conference, Post Script #3

A blog maintained on a personal web page is, almost by definition, self-indulgent. Benjamin Franklin once said, “Either do things worth writing, or write things worth reading.” I doubt that more than a very small fraction of bloggers fall into either of these categories. But it is the irrepressible nature of the human ego that makes the majority of us believe that we belong to the minority. With that sentiment in mind, I present this blatantly self-promoting postscript to my SPIE Microlithography Conference diary.
On day 1, before the keynote talks began, there were the obligatory awards presentations. I was one of three lithographers elected to the rank of SPIE fellow. I am certainly honored by this distinction, and hope to one day aspire to the next organizational grade – jolly good. Also this week, my new book “Field Guide to Optical Lithography” was published (it is shamelessly promoted elsewhere on my website, www.lithoguru.com/scientist/books.html). So on Tuesday morning we had an author’s book signing. You should have seen the crowds, lined up one, sometimes two deep! Fortunately, I had a ready supply of multiple pens to handle the throng. This book will no doubt be a best seller (which first requires a careful recalibration of the meaning of “best seller” when publishing in such an incredibly arcane field).
And finally, since this the third and, one would hope, last postscript to my conference diary, I’ve collected all of the conference blogs up and put them on one page at http://www.lithoguru.com/scientist/conferences/spie_diary_2006.html. For your reading pleasure.

SPIE Microlithography Conference, Post Script #2

In which direction are the prevailing microlithography winds blowing? This is always one of the most fascinating questions to ponder after attending the annual semiconductor lithography geek-fest that is the SPIE Microlithography Symposium. Almost every year some new trend can be detected. Sometimes it is subtle, as a new problem or solution starts to be explored. Sometimes it is blatantly obvious, as when a few years ago the litho crowds deserted all the other lithography talks to attend the first session on 193nm lithography. So how were the winds of change blowing this year? I detected two gentle breezes, not too stiff, but quite obvious.
The first was a breath of fresh air for EUV. Yan Borodovsky in his keynote speech probably didn’t reassure a skeptical audience by showing how Extreme UV Lithography needed to improve by “only” 11 orders of magnitude before it was ready for production. Intel may be the biggest cheerleader for this next generation lithography, but the facts are the facts despite the spin. However, the next day ASML made up for lost ground by giving a well-received review of their EUV scanner development program – not to the believers in the Emerging Lithography conference, but to the life-long skeptics in the Optical Lithography conference. Personally, I still think that EUV technology is the next X-ray Lithography, doomed to failure by unforgiving physical realities. But it won’t be due to lack of effort, and if anyone can make it work it will be the industrious Dutchmen at ASML.
The second breeze felt warmer on my skin (optical bigot that I am), for it flowed along a path that allows 193nm to keep going past the 45nm half-pitch. High index fluids and materials can get you so far. Then what? Double printing, where two larger pitch patterns as interspersed to create a smaller pitch pattern, is getting serious attention. Of course, nobody likes to even think about doubling the cost of an already expensive critical lithography step. And the overlay challenges are formidable. Still, double patterning may first prove valuable as a process development vehicle, enabling device and process learning before other lithographic alternatives are available. Then, efforts towards making the technique production worthy might be successful. There is always huge momentum behind incremental improvements to our existing technology. I am hopeful, if not optimistic, that double patterning will eventually become mainstream.

SPIE Microlithography Conference, Post Script

I am comfortably back in my home on a rainy Saturday afternoon in Austin (this is Texas – we love the rain here). In a couple more days I should be fully recovered from the week’s stresses and strains. At least physically.
It was a busy week. And a big one. My contacts at SPIE tell me the conference had over 4,200 attendees coming to watch about 860 technical papers (compare that with about 100 attendees and 26 papers thirty years ago). That translates into heavy-duty information overload for conference attendees and lots of money for the conference organizer, SPIE. The average cost to semiconductor companies to send a person to the conference is probably $3,000 – $4,000 when the salary of the person is figured in. Accounting for students, technical exhibit only attendees, locals, and people who only come for a small part of the week, the industry is still spending on the order of $10 million just to send people to the conference (though of course the supplier companies probably spend a comparable amount for booths, hospitality suites, and customer dinners). Even for a $250 billion industry, that’s not chump change. Is it worth the money?
The answer is obviously yes. This is an important week – essential even – for lithographers. What is at stake here is significant beyond semiconductor revenue growth. The future of Moore’s Law is in the balance. Don’t get me wrong – I’m not trying to say that if you let more lithographers play in the scientific sandbox the world’s technological progress will be assured. Or even that scaling transistor dimensions according to the tradition of Moore’s Law is needed for progress. Industry advancements will occur whether lithographers keep pushing resolution down for another decade or hit a brick wall next year. But the nature of competition, growth, and progress in semiconductors, computers, entertainment and all of electronics will be affected by what lithographers can accomplish over the next few years. If you’re behind, it will be hard to catch up. But if you’re at the front of the pack, the future is yours to make.

SPIE Microlithography Conference, Day 5 (Friday)

There is one day left in the microlithography conference, but not for me. I am catching a flight back to Austin this morning, leaving the last day of the conference to lithographers hardier than myself. And I leave with mixed feelings. This conference is always amazing, for the collection of people and their interactions, for the scope and depth of technical topics discussed, and for the undeniable impact that this week has on the direction of our industry. But it is also disappointing, because the conference does not live up to its potential. The problem statement is simple, though the solution is not: about half of the oral papers are not worth listening to. Too little technical content, and too much marketing message. How much knowledge do I gain when a resist vendor tells me that “Material A” with “High Index Fluid 1” outperforms “Material B” with “High Index Fluid 2”? How is my job affected by learning that an EDA startup has a new litho model (soon to be available) that is fast and accurate, but they won’t talk about the model, or how they measure speed and accuracy? They should spend money on a booth at the Technical Exhibit if they want to give that kind of a sales pitch.
The difference between a marketing pitch and a technical paper is obviously technical content. And in a scientific conference, technical content is judged by, among other factors, whether enough information is given so that others can reproduce the work. It is one of the foundations of the scientific process. Papers that talk about “resist A” and “tool B” are useless. Papers that simply show off a company’s product don’t fit here. Saying “I can’t discuss that information, it’s proprietary” is an indication that the discussion does not belong in an open forum. The standards for papers at this conference are on the decline. I hope the conference chairs decide that bigger is not always better and try to do something to limit the marketing fluff that surrounds and hides the true gems – the real technical papers – that can still be found during SPIE Microlithography week.

SPIE Microlithography Conference, Day 4 (Thursday night)

Monday night and Thursday night are the poster sessions. But these are unlike any other poster sessions I have seen – monstrous rows of cork boards crowded with people and information stretching as far as the unaided eye can see. You’d think with two and a half hours I could see all of them, but that is just not possible. I gave up many years ago trying to read even some of them. Now I move as quickly as I can just scanning the title and content. If it looks remotely interesting, I give the author a card and ask for a preprint – then I move on.
The problem is that my mean free path is too small. I keep running into people I know, stopping to say hi and talk about a paper or exchange company gossip. Only by rudely professing a preference for posters over people can I carry on with my journey. At least the beer selection for my two drink tickets has improved over the years – not just bud or bud light any more. Somehow that makes everything just a little more manageable.

SPIE Microlithography Conference, Day 4 (Thursday)

I need to be in four places at once. It must be Thursday. It’s parallel sessions hell, with four of the five conferences of the symposium occurring simultaneously. Not only that, but the organizers seem to have purposely scheduled identical topics at the same time and at extreme ends of the conference center. I shouldn’t complain – I need the exercise. In fact it is not a conspiracy to cause me to work off the previous night’s beers, but rather a simple lack of coordination between the conferences. Each conference chair wants to optimize his own conference, which means getting as many papers as possible in his conference, even if some of them really belong in a different conference. Lithography Model Calibration? I saw papers on this topic in every single conference – or rather, I wanted to. I didn’t see them all, because I have not yet perfected the art of being in four places at once.
Funny, this is a conference about making things smaller, but the primary goal seems to be making the conference bigger. I think there are better metrics of success.

Musings of a Gentleman Scientist