Caroline Wardle: I'm going to give you some basic information about obtaining funding that are good for all agencies, and then some things specific to NSF. Jim and Frederica will talk to you about the differences with the other agencies DARPA and NIH and each of us, if we have time, will just mention something at the end of our presentations about a favorite program of ours or something we'd like you to know about a new initiative that is starting at one of the agencies.

Let's talk about general things to think about with proposal preparation. By the way, I'm the Deputy Division Director of the Division of Experimental and Integrative Activities at NSF. This is in the Directorate of Computer and Information Science and Engineering (CISE).

So let's look at NSF. Some very basic things about proposal preparation. The important thing to note about NSF is that it uses peer review. It's extremely important for you to know how your proposals are going to be evaluated. Peer review merely means that your proposals will be reviewed by at least three technical peers. They will be drawn from universities, from government labs, from industrial research labs. There's two ways of managing a review. There's panel review where we bring a group of experts to Washington and we give them 10, perhaps 12 proposals and they will evaluate them all on one or two days. They will be comparing the proposals. The other method is mail review which we use sometimes, particularly if we have a very specialized topic. We would like some additional reviews since it's so specialized, and we literally send the proposal to the reviewer by mail. The reviewer will mail it back to us and we add it to the other panel or mail reviews.

So these next items might look very obvious and reasonable but they're very important. Read the program announcement. There is a program announcement for everything. If there are particular things in the program announcement, make sure that you address those points. Very often in an NSF announcement you'll have the general program description, then it will say, in addition to the regular NSF review criteria, that you will also be evaluated on additional criteria, and it might give you six bullet points. Make sure you address each one of those bullet points. One, for example, might be if you're in a group proposal, it might say you must have an evaluation plan in your proposal. If you omit an evaluation plan, I guarantee the reviewers will jump on that immediately and will downgrade your proposal. So you don't want to have any red flags, you don't want to have things that will cause your proposal to be pulled down.

Submission deadlines are absolute at NSF. If it says a deadline is 5 p.m. by Monday, then you must have your proposal in by 5 p.m. on Monday. We do advise that you send it overnight express because if it gets delayed through their fault, it can be tracked down and then we probably would allow it to come in. But otherwise we won't even see it. It's turned back at the central processing unit. So it's a really firm deadline at NSF when it says deadline. But where it talks about a target date, it is not. The target date, which we don't often use now, says that if you get it in at this point then we'll guarantee to get it reviewed in a very timely manner along with all the other proposals. If you come in past the target date, your proposal review will get very delayed. It won't get in until the next review cycle. NSF is trying to get feedback to you within six months of your submitting your proposal. Therefore, it's pretty well moved to panel review because it's much faster. So most of our submission dates are deadlines.

This is really important. Before submitting your proposal, you should talk with your funding agency. We like to talk to you. Come and talk to us about where our program priorities are. We can give you advice on how to write your proposal. If you don't know exactly who to contact, but you know somebody at the agency, ask them who you should talk to. Send them an e-mail, a paragraph describing your research area and they will point you in the right direction. And if you are new to writing proposals, as many of you will be, contact an experienced professor at your institution. Ask them if they would read through your proposal and critique it for you. And don't give it to them two days before you have to submit. Give it to them two weeks before. That will give you time to think about the comments and perhaps make some modifications to your proposal.

Now, this one I'm only going to put up for two minutes because it's very basic. When I'm a reviewer, I want to know: what are you going to do? How are you going to do it? Why is it important? Is it feasible? What is your unique contribution? Why are you the best person to be doing this and, very important, what are others doing in this area? Many program managers have reported reviewers saying: "You know, this looked as though it was a good proposal, but they didn't give me any indication that they knew what was going on in this area. There was no literature search, there was no description of related work." This is very important.

Let me give you some proposal presentation hints. Again, you might think these are obvious but I can assure you having read many proposals, they're not. Present your ideas clearly and succinctly. Don't forget you're scientists, we're scientists, the reviewers are scientists. They are used to reading the scientific style. Nothing irritates me more than a proposal that has grammatical errors and spelling mistakes in it. Please, if you tend to make spelling mistakes, put your proposal through a spell checker and ask a friend or a colleague to read it for you.

Provide adequate technical explanation. You will have in your reviewers some who are experts in your area. You will have other reviewers that are experts in related areas and have a broad general knowledge but may not know every little thing about your area. Make sure, just as you would in a scientific paper, that you provide some explanation as to what you are doing.

Budgetary matters. All universities have research offices, either an office or a person. These people know the various ins and outs of the agencies which are all different. They will help you put the budget together. They will be very helpful. I've seen many times, excessive budgets that really irritate reviewers. You do not want to irritate reviewers. I'll give you an example. I have seen proposals that ask for equipment and they come in asking for a big graphics engine and six workstations. And you look through the proposal and you find one professor and two graduate students. So you think, how could these three people use six work stations, and there's no justification in the proposal for that. So that's very irritating. That's what we call padding the budget and it comes out straight away. But you don't want to irritate the reviewers. Come in with a good, tight budget. Justify each part of the budget. Justify release time or summer money, justify the equipment, and what will come across is a very nice budget and proposal.

And this is very important. You're selling to the reviewers. In NSF, we're peer review driven. If you don't get past the first hurdle of reviewers, you are not going to get funded on that proposal. So remember you are not sending it to your colleagues who perhaps know your area very well, you are writing your proposal for the reviewers.

I thought you'd just like to see some statistics from NSF to give you an idea of how competitive it is. CISE had just over 1,700 proposals last year of which just over 500 were awarded. So our funding rate, what we used to call the success rate, is the number of awards divided by the number of proposals times 100, that's 30%. So we fund 30% of proposals. That's gone up a little bit. The NSF averages the same and it used to be about 28% so it's going up. The average annual budget, as you can see, is just over $100,000 and most proposals now last between two or three years. So you're getting roughly $100,000 each year for two to three years.

Another way of looking at this funding success rate of 30% means that your failure rate is 70%. So what is important to point out, is not give up if you do not make it the first time. It's unusual for somebody to be funded the first time. You really have to learn how to write proposals. As you can see, if 70% of the proposals are turned down -- we use the word, declined -- then it's a really competitive situation. There are budgetary limitations that you have no control over. I mean this year my program has received a budget cut. I couldn't control that and nor can you but it means I will not be able to make as many awards as I had hoped for. We try to make as many as we can but there are budgetary constraints. So very often when I say I am going to fund you but I can't fund you at the level at which you ask, I will try to negotiate a slight reduction in budget.

NSF priorities exert influence. Each year we will have a different priority, different themes at NSF, so this is important for you to know about. You'll find out about those if you contact your program director. And this is very important. Look at those reviews. They'll give you good advice as to how to improve your proposal, rewrite it, and resubmit. I have to tell you the story of one institution that came in four times, four years in succession. It got turned down three years but kept coming back. It improved the proposal every year and it won $1 million on the fourth try. So we do encourage you, don't give up. Always try to come back.

There is just one more thing that I want to say. I want to tell you about something new that I'm doing. I am now working on a new program under the Presidential Initiative IT2. We're hoping that NSF will get a lot more money next year and I have been asked by Ruzena Bajscy to put together plans for programs for women and minorities in Information Technology. You may have been reading in lots of journals and newspapers that there's a shortage of information technology workers which should make you feel very happy. There are lots of jobs out there, but we're very interested in getting more women into the field. Some of you may not know that the number of women, for example, receiving Bachelor's degrees, which is where we get our big cohort, has been dropping steadily since 1986. In 1986 we had over 15,000 Bachelor degrees in Computer Science going to women. There are now less than 7,000 and it's still dropping. So we're looking at this situation of women not going into computer science. So we need to fund some research studies to try to find out why this is happening.

So what we're going to be doing is running virtual workshops, two electronic discussion groups on the Web. I expect that these will probably get going about July and August, so please keep an eye open. We'll post it on the NSF Web Home Page and we'll put it on the Sister's network, for those of you on that. We'll alert you to that. Please join in. We'd welcome your suggestions and ideas of studies that we can do to do to try to find out why women are opting out of computer science and how we can get them back again.

So now Jim Cassatt will come and talk about NIH. And we'll have all the presentations first followed by questions and answers for the remainder of the session.

James Cassatt:

Thank you very much, Caroline. Everything Caroline said about NSF, please keep in the back of your mind because most of it including things like proposal preparation, what to do if you fail, reading reviewers comments, not taking things personally, etc., etc., also applies to NIH proposals. So I need not repeat that. I am told this is the first time that someone from the National Institutes of Health has actually come before you and so this is going to be an experiment.

The National Institutes of Health is a very confusing organization because it isn't a single organization. There are something like 20 different institutes contained within the National Institutes of Health. Each has its own separate budget and, therefore, the things like success rate, average duration of award depends very much on the institute. Each has its own particular mission and initiatives. So it becomes very complicated. Most of the institutes are focused on some disease, e.g., the National Cancer Institute, the National Heart, Lung, and Blood Institute, the National Eye Institute, etc. You can guess what their missions are related to.

I am from the National Institute of General Medical Sciences and our mission is to fund basic biomedical research. Sometimes we call ourselves the NSF of NIH. When we talk about our mission, that is to fund biomedical research, you should take that in the broadest possible sense. And, in fact, a lot of what we fund at the National Institute of General Medical Sciences has implications for direct medical benefit 20 years away. And we take a good deal of pride in that.

Why do we want you? Biology is changing and it's changing very, very rapidly. You hear buzz words like post genomic. Within two or three years the sequence of the entire human genome will be known. You hear words like post reductionist but since I've been in science 25 or 30 years the approach has been to pull things apart and understand how the details of each component part of something -- and this is generally what we're talking about in biology, an organism all the way down to the cell and down from the cell down to the component parts of the biomolecules. Every group that we pulled together in the last two years has said you've got to start putting things back together. We're talking about new ways of doing experiments. In biology, you did experiments, you got some results, you analyzed the results. There is new chip technology that is being introduced very rapidly so that you can do one experiment and find out what is happening to every single gene within an organism. So information is enormous in biology. Biologists don't know how to handle that information. This is a problem in computer science. It's just wide open for computer scientists. We've had, again, any number of panels when Harold Varmus, our Director comes and talks about the need to bring people into biology who are not biologists. People like physicists, chemists, mathematicians, computer scientists. We desperately need this kind of new expertise to be applied to biology to continue the advances that we've made.

Now why should you want us? The NIGMS budget is over a billion dollars. The NIH budget is probably, what $12 billion to $13 billion? The success rate for NIGMS grantees is somewhere between 35% and 38%. Contrast that to 30% you heard on the last slide. Our average size of a grant is around $220,000 total cost. Our average duration of a grant is four years and for new investigators, of which I hope there are many in the audience, we generally provide five years worth of support. Caroline was talking about someone needing six work stations. We're commonly handing out Pentium farms. It's amazing what you can do with $50,000. You get a lot of dual processor Pentiums and you've got a little mini super computer sitting in your laboratory.

So we actually have things to offer. Now, if I turned you on, you want to know how to apply and what happens. NIH requires a form, a 398 form. It's actually a kit about this thick. If you want to see what it's like, just go to your sponsored programs office, we send out gazillions of these things all over the country. They will have the kit. Or you can go to our NIH web site and you can download a copy in PDF format. We have not quite gotten to the point of you being able to download this in nice format that you'd fill in by computer and print it. No. We're not in the 20th Century yet.

What you will find, though, when you go and look at the 398 kit, there's a lot of forms that you fill out with budget and everything else. But the most important part is the research plan. This is very similar to what you would submit to NSF and it's very highly structured. Not following this, you do at your risk. It requires a statement of your specific gains in one page and this should answer it because here is what I'm going to do for the next four or five years. NIH study sections are used to seeing hypotheses. We're trying to get them out of that mode because much of the science that we're seeing is not hypothesis driven but they're used to seeing this. So, again, you should state your hypothesis.

The next section is what is called backgrounders significance. It should answer the question, where does this fit into the overall picture? Why am I doing this?

The next section is preliminary results of progress report. A progress report, if you'd had a grant and you're applying for a competitive renewal, but the preliminary results are, again, things that people are used to looking at. Clearly, if you're a new investigator you're not going to have as many preliminary results as someone who is seasoned. Nonetheless, it should answer the question, here are the things that I have done to show that my approach is feasible.

And finally the method section. Here is how I am going to do this. And this has to be done in 25 pages or less. You can send in some appendix material. Again, everything that Caroline said about how to prepare a proposal falls in -- is applicable here. NIH actually has regulations on font size. Just because they let you use a certain font size doesn't mean you should use it. Picture yourself as a reviewer who is going to review this thing at 10:00 at night after he has put all his kids to bed and he's dead tired, small fonts stretched to the margins, even if allowed, do not make for pleasant reading and do not make reviewers happy.

With respect to peer review, again, we use peer review exclusively. All of our review would be, I guess, the equivalent of NSF panels. When something comes in, the application will receive two assignments. Assignment One is assigned to a study section for review. These study sections are based on scientific areas. There are about a hundred of these study sections within NIH and the study sections will rate your proposal and rank it within the study section and we use those rankings as the primary parameter in making a funding decision. If your grant does well in peer review, basically, if it does well enough it will be funded. If it doesn't do well, it will not be funded. We have all sorts of programs, etc. Ninety-five percent of our applications that come in are investigator initiated and don't respond to any of our program announcements. So peer review is the single most important factor whether your grant is going to get funded. Exactly what Caroline has said.

Secondly, it is assigned to an institute for funding and each of the institutes has its own particular mission. So if you're doing some sort of biological modeling related to cancer, it's going to go to the Cancer Institute. These two decisions are independent. Review is done by different organizations than does the funding.

Now, if I'm done my job you're all saying biology is really exciting and even if I haven't done my job you can look in the job wanted sections of science and you'll see every company is looking for bioinformatisysts, computer scientists, etc., who want to get into biology and you find that these are a rare breed. But if I've done my job you're sitting here saying, well, I've never taken a biology course in my life. What do I do? Well, we actually are worried about this and it's going to be approached at various stages in one's career. We have collaborative mechanisms. For example, you're computer scientists, you're faculty, and, hey, this turns me on. We have ways of supporting you if you can collaborate with someone who already has an NIH grant. We have something called mentored K awards. Again, you're computer scientists, I really want to do some biology. Let me find a mentor who has an NIH grant, presumably. You can apply for this -- it's a $25K award and this is brand new. It's just been announced and I'm getting phone calls now about this.

Suppose you're a faculty member, you've just gotten tenure. Congratulations. And now you can actually do what you want to do instead of trying to worry about trying to do safe projects to ensure publication. Biology turns you on and you really want to collaborate with a biologist as part of your first sabbatical. We have so-called senior fellowships, what we call F33's. Usually the sort of thing that happens is the university will pay half your salary, we can pick up the other half. Again, this would be -- like any other fellowship application, it's written in conjunction with the person you're going to do your project with, you have to have a project that has a recommendation, and so on. These go to study sections and are reviewed, etc.

Well, let's say you've just completed your Ph.D. in computer science and you'd like to go to work with someone in biology. We're happy to support you. We recognize the fact that your proposal may be a little naive. We have a fair amount of fellowship money and, again, these are reviewed by peers. The mechanism is straightforward and we're on the lookout for people with backgrounds other than biology. I was very pleased. I signed a bunch of 15 fellowships very recently and only 3 of those were from biology majors.

And finally, at NIGMS, we don't fund pre-doctoral fellowships with some minor exceptions. All of our pre-doctoral training money is sent to institutions. So, again, you've just gotten a Bachelor's degree or you know someone who's got a Bachelor's degree in computer science and wants to do biology. There are training programs that we support and other support that will take someone who has no biology background whatsoever but has interesting credentials and turn them into computational biologists. And we have a program, for example, in molecular biophysics and many of these are very successful in taking people with no biology backgrounds and training them in the areas to make real contributions in biology, taking advantage of their quantitative backgrounds. Boston University has a program funded by NSF, actually, in bioinformatics. And the emphasis there is to take either biologists who know no computer stuff, no computer science, or take computer scientists, mathematicians who know no biology and bring them to a common point where, again, they're able to make contributions.

Caroline Wardle: The program announcements for NSF are all on the web now. So you can go to NSF and look up computer science if that's where you're interested. Is that the same with NIH, Jim?

Jim Cassatt: The NIH has a formal mechanism for announcing everything. The NIH Guide to Grants and Confex. Anything that's official that NIH wants to announce appears there including program announcement. This is available on the NIH web site and is searchable.

Frederica Darema: DARPA has a broad area agency announcements and they're all posted on the DARPA web page and on the CBD. I would suggest the easiest way is to periodically peruse the DARPA web pages for the different offices, to find out what their new program announcements are there. Also, DARPA has small Business Innovative Research program and projects, and I will talk a little bit more about them in my presentation. They are also announced on the DARPA web pages under the SBIR office there.

I'm going to hand this over to Frederica Darema now.

Frederica Darema: Good morning. I would like to present to you a perspective for submitting proposals to DARPA, the Defense Advanced Projects Research Agency. Getting funding is the title of this section, but most important is what I have in the parentheses underneath: ensuring that your ideas get funded. And that's more and beyond submitting proposals and writing good proposals, and I will talk about these issues also. So, first, I will give you a perspective from DARPA but also I will provide a comparison of the differences in the process with those of NSF. I was for two years on leave from NSF to DARPA, and so I will tell you a little bit about some differences between DARPA and NSF opportunities and proposals, and how funding decisions are made. As I said before, I will also talk about some additional items which pertain, in a broader sense, to "getting your ideas funded".

NSF is an agency that traditionally announces programs, where the program content is driven primarily by the scientific community input. The NSF programs are sustained in their for a long time, same context for a number of years. At DARPA, the programs that are announced are much more agile in their scope. The Program Manager that comes to DARPA is expected to have a vision for some of the technical directions in their own particular area, and also not only have a vision but have specific technical ideas of how to go about in accomplishing that vision. Having that kind of vision, the Program Manager (PM) creates a briefing. The briefing is very much like writing a proposal. As the PM you really has to convince your upper management that it's worth for them put several tens of millions of dollars into that program, so you can dispense it to researchers that you feel that will accomplish this vision. So, like when you are preparing a proposal, you have to demonstrate that there is a need for this work.

Certainly DARPA as the Defense Advanced Projects Research Agency has as the primary mission to develop technology that will help the DOD, the Department of Defense. However, -- in a sense -- there is no barrier between developing technology that supports the Department of Defense and (the research and technology) having a broader impact. And we have a lot of examples of that. For example, the Internet and VLSI technology. Work leading to the internet was spearheaded by DARPA many years ago. The VLSI technology efforts also started by DARPA, again, many years ago - that's about 20 or so years ago. Now going back to specific programs: The program manager (this is what the people that manage DARPA programs are called, in distinction from NSF where they are called program directors) at DARPA is expected, also, to develop a scope for the program, because the idea is not to fund the broader needs of the research community. DARPA, does not have the approach that we fund a broad research area to foster a research community. DARPA funds a specific research area and with a given set of goals.

The other item that a Program Manager has to include in the program formulation is: what are the milestones for this program? What are the key technologies that will be developed in intermediate steps of the program and to accomplish the final task? And, based on these milestones, the deliverables and the scope of the program put together, the PM has also to determine what is the required budget to accomplish this task. So, constructing DARPA programs, has some similarity with the process of puting together an individual proposal. And as Caroline said, you've got to sell your proposals to the program managers and to the reviewers. Likewise, the program manager at DARPA needs to sell their program, their ideas to their office director and to the DARPA director and deputy director. And once the budget is allocated, then a program announcement, or the equivalent of a program announcement, is made. It is called a BAA, Broad Agency-Announcement.

Typically, the programs at DARPA last three to five years. Some (programs) a few years beyond that, but there is always this notion of agility at DARPA, that you don't support programs forever. In other words, you address a technology area, you make a fundamental change, a quantum leap in the technical scope of the field, and then you move on to address another area. And, of course, you don't abruptly phase out support, for example efforts towards the internet started about 20 years ago, and DARPA has sustained networking research efforts for a number of years in the interim time. But the technical emphasis, is changing over the years. Because you have to address different technical challenges when you enter a field, when you create a field, versus when the field matures.

Typically awards are made for three to five years. They are made to academia as well as to industry, and that's a distinction from NSF, where typically and the majority of the awards is made to academia. Sometimes the programs are formulated in a two-phase fashion - the first phase two to three years to demonstrate some of the feasibility - because sometimes the area that is being fostered is not something that has been addressed by the research community before. So before deciding to allocate a tremendous budget, you want to demonstrate whether some of these ideas will pan-out at all, or have a possibility to pan-out. If the first phase proves promising, then a second phase follows. After a program is approved, a BAA is issued, and then proposals come in, are reviewed, and some are funded. There is a possibility, in subsequent years to have additional BAA's, in order to fund work to fill gaps - where the Program Manager deems that the first round of proposals hasn't addressed appropriately or adequately the (research) area.

So as I said before, the "mechanisms" are the BAAs, and they are posted on the DARPA web pages and on the CBD - this is the formal process that the government announces such opportunities. Sometimes, or actually not sometimes, I would say quite frequently, there is a briefing to the community on the subject of the BAA. This briefing is done prior to posting the BAA, because the BAA is essentially a call for procurement, so there is nothing that we can say to the prospective PIs after the BAA is issued, because you don't want to give an advantage to one PI versus others.

Also, besides the BAA's, there are specific announcements by the SBIR program -Innovative Research programs for small businesses. These are announced in addition to the BAAs. Proposal deadlines: Typically the BAAs are announced in the beginning of each fiscal year, because then is known the funds available for program, for that year. Typically the proposals are due 60 days after the BAA is announced - sometimes a bit more time than that, but 60 days is the minimum required.

The proposal sizes: they vary. Typically from 200K/year, but most of the proposals are higher than that - up to more than a million dollars per year. The specific numbers are not public, so I'm not going to say more about those numbers.

The proposals are reviewed by panels. There are no "ad hoc" reviews like NSF, and the panels consist from other DARPA program managers and program managers in other agencies, industry representatives, as well as people from academia - perhaps academics that come to serve in the agencies as IPAs. Proposals are binned in five categories; like: proposals rated "must fund", or proposals deemed "selectable" in the sense the research that is proposed is very high quality and they can be funded if funds are available, and other proposals are put in the category of "reject".

The proposers are notified by mail on the outcome of their review, and they may receive a briefing summarizing the gist of the evaluators comments. DARPA does not provide individual reviewers comments . in distinction with NSF, where for example the PIs see the actual reviews for their proposal. The proposal success rate is about 10% to 20% depending on the submissions and the amount of funding available.

Some other differences from NSF's and NIH's funding process that you heard from previous presentations: the proposals funded from DARPA are actual contracts. The PIs are contracted to do a specific amount of work which they have proposed in the proposal. The projects have specific deliverables. They have milestones, research accomplishments, and the technology deliverables, and they are payable in installments as the contracted tasks are completed. In other words, every quarter the PIs submit what their milestones and accomplishments are, and then the agents (DARPA) will send the next installment of money to the (PI's) institution. So if you don't do the work promised, you don't get the remaining funding. (As I said before, the products of the research are of value to the DOD but, of course, as we know in many instances there is much broader value to the society at large.)

Also, the projects are reviewed periodically. That's another difference between DARPA and NSF where you are given actually a grant. When you've gotten your award from NSF, unless you do something very bad, you're not going to lose your grant. Whereas, at DARPA the projects are reviewed periodically. That's good in many ways. the PIs have to provide quarterly reports and - I know it's a burden, but you should consider it as positive because it provides an opportunity for the program manager that has a broader view of what happens also in other research projects to help some in shaping your project. But also the projects may be terminated! DARPA reserves the right to terminate the project. That's can be due to unsatisfactory performance. It can be done due to budget cuts. And it could also be done due to redirection of DARPAs needs.

Of course for the majority of the projects, they are funded to their completion, and at the end there has to be a report submitted on the deliverables. Sometimes the deliverables are actually technology products. Not commercial products, but products that can be used by DOD for example.

So these are some of the differences between NSF and DARPA. Here are some more. I mentioned earlier that most of the programs at NSF are community driven. NSF has workshops and invites the community to tell the program directors (at NSF) and NSF's various directorates, about new research directions and new vision - what are the exciting research area for NSF to fund? At DARPA, new directions, are mostly generated by the vision that individual DARPA PMs have. And as I said, NSF funds research areas at large, like: database research, compiler research, robotics and so on. At DARPA, a lot of this basic research is also funded but has a specific pull - to develop a specific technology. It's application driven. This is not bad. Applied research or application driven research does not mean it doesn't have basic research content. I want to make that very clear. And we have seen it from the impact that DARPA has made. And as I mentioned before, DARPA has contracts, NSF has grants, and DARPA has specific directions versus free thinking. The deliverables for DARPA - products of use to DOE ; for NSF the "products" of the funding are publications.

The DARPA PI is typically selected among researchers that have demonstrated strong research capabilities, so it is very hard for someone that has just finished their Ph.D. to submit a proposal to DARPA and get funded - certainly it can be done but that's not typically the case. NSF, has a lot of mechanisms and programs to help researchers to build up their professional track record; NSF gives CAREER awards, POWRE awards and so on. DARPA does not have those.

Some thoughts for writing successful proposals: a key ingredient and first of all are your own novel, great, innovative ideas. The rest of the points here are items that you've heard from the other people in this panel. You have to define the scope of your proposals. It cannot be unspecific, like: "I'm going to cure cancer", but not say how you're going to do it. In the executive summary of the DARPA proposal, you have to say what you're trying to solve, why this particular project, why is the time now to do it, what is the "quantum leap" that you're going to achieve. And also, as the other speakers said, why this is important with respect to research that is being done already? What is the differentiating, distinct aspects of your proposed work . Why is your method is better. And certainly you have to substantiate why this is responsive to this particular BAA. There's a very small number of proposals that are accepted on an unsolicited basis but this is a very, very small amount of the funding.

SIDE B

Frederica Darema: The proposals (submitted to DARPA are extensive) . they are about three quarters to an inch thick, and the reviewers receive a dozen of those. Reviewers spend a lot of time in evaluating the proposals. So don't make it more lengthy than needed. You don't gain more by padding it with words.

Besides writing good proposals, there are other, related mechanisms which are very important in getting your work funded. Like: Getting known to the PM - go and talk to the PMs. Go and talk to them about your ideas. You can do that, as I said before, before the BAA comes out - you cannot talk to them afterwards. Keep in mind, the Program Manager is your friend. I mean, we in the agencies really want to help you to do outstanding research - because this is how we get to accomplish our objectives (of funding good research). We announce a program and ten years later this great research has been done (supported by that program) and is attributed to this program. So the PMs are very eager to help each one and every PI to succeed. Keep that in mind.

Another item: get involved with writing proposals. Even in graduate school, when your advisor writes a proposal, volunteer to help. Ask other peers to help you -- to give you feedback on what you're writing. ( - These are all things that were addressed before by the other panelists.) And consider your feedback very seriously so you can try next time and be sucessful. And if you're junior academic, try to collaborate with more senior researchers. But also have in mind that you should establish yourself very quickly on your own.

Don't assume that people understand first time what you want to say. When you tell people something, either in a presentation or in a proposal, "tell them what you are going to say, say what you're saying, and end by saying what you said!" And the more novel the idea is, the more this kind of approach is necessary. Present your ideas to multiple forums so you can get feedback and document your ideas. My favorite saying is the Latin maxim: "verba volent- scipta manent" - means: "The words fly, the writings remain". You want to be able to have people tie the idea back to you, not forget about it.

Other things: Be on top of your field. Be aware of important trends now in getting funded - like multi-disciplinary research. Jim mentioned that in a wonderful example, with the efforts that NIH does, but this happens with other fields. So become familiar with some other areas, because there is a leverage and opportunity in the boundaries of the research areas. Network. Know other PIs. Seek mentors - and I think other presentations may talk more about this.

Start publishing early. Be active in the broader community so you can become known in a broader community, where the reviewers, the ones that will review your proposals, come from - it helps always when they've heard about your name before. Or have seen your work before. Go to conferences.

And I want to put here a plug for people's involvement in the agencies. Like, when you are called to review - ad-hoc reviews or panels - please accept it. The best way we can support proposals and make the case for a good proposal, is to have a good set of reviews and careful reviews. That is also good for the proposers whose proposals are not successful - they need to get meaningful feedback on that. So don't shun this duty.

Also participate in workshops that are used to create new initiatives in the agencies. This is more done at NSF but also at DARPA there are such opportunities and new program briefings.

You should have in mind that, eventually in your careers, it is a good thing to come to the agency and be a program manager or a program director, because getting your ideas funded, is making a case that your technical area is very important and can have a broad impact presently and in the future. And a very good mechanism to do that is to create programs. See that it is an opportunity. You don't leave your research but you have an opportunity to make a broader impact in your area.

And having said that, I want to put a plug for a program that I just announced coming back from DARPA: the Next Generation Software program, it pushes for some new ideas in several technology areas, like: performance engineering of hardware and software, new compiler technology, new application composition technology. Look for it in the NSF web pages.

Caroline Wardle: Thanks, Frederica. Feel free to ask us anything. You can direct it to a particular agency or we'll just ask for any volunteers to answer it. Anybody have any questions, anything to do with getting funding?

Audience: I have three questions. Could you please compare the cancellation policy on NIH and NSF projects. You said that DARPA has a mechanism for canceling projects. The second one is, how can I meet project managers at conferences? How do I know who is who? And the third one is about feedback. Could you compare the feedback that's given to the projects.

Caroline Wardle: Let me quickly answer for NSF cancellation. Frederica mentioned we give you a grant and it might be anything from one and three years. You have an annual report to do and as long as you seem to be making progress, we keep continue -- we will continue the funding. If we think you are not making progress, we can and come do a site visit. We rarely do this for individual grants. We certainly do it for the larger group grants and sometimes, it doesn't happen very often because people usually do do the work that they say they will, but occasionally we have gone in and the site visitors have recommended that the funding be terminated and we have done so. So we just stop any continual funding on that project. But where it really hurts you is that when you come in with another proposal, it is usually known in the field, as well as by NSF, that you did not do good work on the previous funded project. Your credibility has been hurt for future projects. But we are not draconian as DARPA is. We do give you the grants and give you a little leeway.

Conferences. The best thing is to get in touch with your program director or manager before a conference is coming up. Find out who they are and whatever agency and just say, oh, by the way I'm going to be at so and so conference. Will you be there? And then you'll find out directly that way. That's the best way. Make contact with them before hand or if you're not sure, just contact somebody you know at NSF and, as I said, ask them who is the person you should speak to.

NSF sends you all reviews that are done by external reviewers. We just remove the name of the reviewer because they are confidential but you will be sent them and it usually takes a couple of months after the review took place.

Jim Cassatt: If I could answer the same questions. In terms of -- you get your money, you actually apply for your grant every year even if awarded for five years but the review is done by staff and is fairly perfunctory. As long as you're making good progress, nothing happens. As far as meeting program managers at meetings, you'd better invite us. We're not going to come to computer meetings, frankly. We do go to a number of meetings more related to the biological sciences. The third question was review. When you submit an NIH grant you will get back in -- you will get two things back rather quickly -- one thing you get back fairly quickly. That is the grant is reviewed by a study section. You will get back rather quickly after that study section meeting, a postcard which tells you the score, and then there will be a number there that you can call, usually my number or somebody else's number. You say, I got this score. What does it mean in terms of funding? And people will say: I can't make any guarantees but ., forget it, on the borderline, hey, it looks really good.

With respect to feedback, about six to eight weeks after the review you will get a summary statement. The summary statement contains the verbatim comments of the reviewers, sometimes slightly altered to protect confidentiality, and sometimes you will have someone else's dissection for whom English is not his native tongue and the person who writes these things will change it slightly. There will be a summary paragraph which summarizes everything. This is why the proposal did this. Reviewer One is very enthusiastic but let's face it, in discussion Reviewer Two's comments were taken more seriously. Unlike NSF and, perhaps, even DARPA in the back will be a roster of the people who are on that study section. NIH believes in openness. You have a right to know who was on this dissection who reviewed your proposal. You have a right to know by the Freedom of Information of anyone on the outside who has seen your proposal. You can request that information under the Freedom of Information Act. You have a right to know the composition of the study section prior to the review.

Caroline Wardle: Do you want to add anything Frederica?

Frederica Darema: The only thing I want to add is (because I talked about the termination of grants and the criteria) is look at there DARPA web pages and see who are the PMs that are supporting programs that you are interested in in your technical area, give them a call, send e-mail, and ask them if you can go and meet them in Washington, or you can ask them if they will be visiting your area, your university, your conference or something like that. PMs are willing to meet with individual PIs.

Caroline Wardle: If you have a number of people in your university or say from surrounding universities and you to say, look, I'd like to get together 9 or 10 people, or even 5 to 6 people, could you come and spend an afternoon with us. That's something you can ask, too. If we can fit it into our schedule we will also do that. More questions?

Audience: This is a question for James Cassatt. I was wondering if you could clarify the priority scoring and the percentile rankings and also, too, the funding levels because I think you gave 35% to 38% for GMS but I think the funding at the 20th percentile in terms of proposal ranks.

Jim Cassatt: What happens at a study section is grants are reviewed, reviewers present -- they're assigned to usually two or three reviewers to present detailed reviews or discussion at the study section. The study section, each member then, is asked to vote a priority score from 1 to 5, 1 being the best, 5 being the worst. The scores are averaged, multiplied by 100. We give you a score that is from 100 to 500, then these are percentiled within the study section such as the 1st percentile is the best, the 100th percentile is the worst. The success rate, which is what I gave which is different from a percentile cut-off, the success rate is -- for NIGMS, that's our institute and that's different from all of NIH, our success rate has been better than most every other institute for the last ten years, is between 30% and 35%. This does not translate into a percentile cut-off. And those of you who are more mathematically and algorithmically-literate than I am, you can probably figure out why. Our pay line, this is the percentile cut-off to which we go by vote, is somewhere around -- right now it's about 27, 28. As we get towards close to the pay line we call this a gray zone and then that area we, as program managers, program directors, everyone has their term for these heads of -- leeway into what we do with those proposals.

Caroline Wardle: I think I would just like to add one more thing that I don't think I mentioned with NSF. I believe Jim, and correct me if I'm wrong, that a difference between NSF and NIH is that the program managers at NSF have a little more leeway in the funding recommendations than those at NIH. Our reviewers do a similar thing and they put proposals in three categories. Highly recommended, recommended, not recommended. But they do not have to fund them in rank order. We typically don't ask them to rank order but merely to put them in those categories. Usually we find that the highly recommended are outstanding but we can actually fund some that might be regarded as lower, that are in the recommended category. And very often we will do it for a number of reasons that is within the mission of NSF. One, we've talked with that PI. We happen to like what they're doing, they're relatively junior, haven't had a lot of experience in writing proposals and we really like to help young and new investigators. So we will often recommend funding for somebody who didn't get into the very highest category. We also have a responsibility to fund a geographical distribution and also large universities and small universities so we try to have a mix. So we will very often go out of order and recommend funding because we think there are some special reasons why we should fund that person. Am I correct in thinking that you were really bound to the rank order at NIH?

Jim Cassatt: No. No. We can actually do anything we want to. Practically speaking, it requires strong justification to not fund someone in the "highly ranked" category and you can translate that into numbers if you want but certainly we superimpose on top of the ranking our own programtic criteria such as we really have a strong push to fund new investigators. We look very carefully at how much other support this person has. How does this -- what we call a program relevance, in other words, is this person doing the same thing everyone else is doing but with a different twist or is this something really new and exciting which may not have scored quite so well because it is new and exciting. You can picture all these things that we do. Study sections actually also look at these things as well and so while we can do a lot of stuff, if you were to say, what do we do, probably effects probably fewer than 10% of the grasp, just on a numerical basis. But we do take these things into account.

Caroline Wardle: Very good.

Audience: I'm from Compact Systems Research Center. I have two questions. The easier one first. Is a new investigator necessarily a young investigator?

Jim Cassatt: No.

Caroline Wardle: I regret saying that. I should have young, I shouldn't have said new.

Audience: So somebody who hasn't gotten grants before?

Caroline Wardle: Why don't you answer that, Frederica.

Frederica Darema: I think this is an unfounded rumor. I think the amount of the award is commensurate to the idea and it is possible a senior PI or a person who has a lot of years of experience to think of something very novel that only requires a small amount of money to demonstrate the concept. It is absolutely delightful if that person submits a proposal like that. Nobody would frown against it. There are no such issues - I mean, this is absolutely a false rumor.

Jim Cassatt: I will just make a comment. The National Academy has just announced their new electees for 1999. It did my heart good to see Joe *Felzenstein* listed. Most of you don't know Joe Felzenstein, probably none of you do, but he's a population biologist, very much into computers, and he runs a very small operation of about $70,000 a year of external funding.

Caroline Wardle: Are there more questions? Feel free to ask very basic questions. We'll answer anything we can.

Audience: Is the NSF presently funding robotics research?

Caroline Wardle: Yes, in fact it funds it in the CISE directorate, in the IIS division. There is also some robotics work, I think, in the Engineering directorate.

Audience: I'm from the University of Wisconsin. I have a question about submitting proposals to different institutions or different funding agencies at the same time on similar ideas. How does that work? I mean, if you want to be sure to get funding on something, can you submit it to two agencies?

Jim Cassatt: NIH doesn't care but we -- our grant things are changing but there used to be a place called other support. All we want to know is know about it. NSF can comment. It's idiosyncratic to which part of NSF you're from, I think.

Caroline Wardle: NSF has a rule that if you have submitted the same proposal to another agency, you must list all proposals at the end. In fact, it has a slightly different -- an answer to a different question is you cannot submit the same proposal to two different groups within NSF. That is not allowed.

Jim Cassatt: I do know in the biological division, Mary Clutter said, no, we will not accept duplicates to NIH.

Frederica Darema: This is true also for DARPA. You cannot submit exactly the same proposal you have submitted to another agency and as Caroline said, you have to disclose all your pending proposals. In other words, proposals that have been submitted and at the time that the program ends or the program director is going to decide to make an award, they will -- at least I would and I'm sure most people do that, they will ask the PI what is the status on this other awards because we don't want to fund duplicate work. I mean this is a problem, frankly, because the money is not enough to fund all the good work and this should not be aggravated by funding duplicate work.

Jim Cassatt: Actually, it's illegal to accept federal money for two projects -- from two sources for the same project. In fact, if you were to get away with it and they would audit, you actually have to give money back.

Frederica Darema: That's also true.

Caroline Wardle: I have often had for some of the larger grants, PIs having a pending proposal with DARPA and they've announced it and said it's complimentary. So if it reviews well in my panel and I contact the DARPA program office and I say, you have a proposal from so and so, I've got one too, and then we discuss it. We find out how it is doing there because it's important to me. We have to make a decision whether my -- the work in my proposal is dependent on the work in the DARPA proposal. So if I want to make an award but DARPA doesn't, then mine won't fly without the DARPA funding. So there are lots of reasons that the agencies do work together. We talk to the program directors in the other agencies to make sure that it makes sense and everything fits.

Audience: I'm from John Hawkins University. I was wondering how often from these proposals you ever saw research directions change along the way and, if so, what happened with the funding?

Jim Cassatt: At what point are you talking about in terms of research direction?

Caroline Wardle: Do you mean when you received a grant and then want to change

Audience: I mean after you receive a grant and perhaps in the three years something happens, they follow another lead instead of the initial primary goal.

Jim Cassatt: At NIH, say we give you a five-year grant and let's say in year two you want to have a significant change in what you're doing, you actually need permission from us. Now we're very understanding. It's a matter of degree. I mean you have to start with something that was way down here that you were going to do. If something takes off, fine. Great.

Caroline Wardle: Same with NSF. As Frederica pointed out, we like lots of curiosity driven research so if you've got some idea that's come out of the side of your research that you think is much more interesting, talk to your program director and they will advise you as to whether that's fine. As long as they say, yes, then it's fine.

Jim Cassatt: One little nuance on that though is at least at NIH every four years, five years you have to come back in for competitive renewal. There are a couple of things reviewers will look at. How much progress did you make during your previous grant period, as Caroline said. They also want to know, hey, you put down specific things, one, two, three, four, five. Which ones did you accomplish. And if you decided to go off on a different track which was really exciting, say so and explain why you didn't do these things but you really went off in this direction.

Audience: I'm from Northwestern. I know Caroline mentioned a new call that was based for women and minorities in computer science. Ca you address how those grants are reviewed as opposed to the general grants? I've actually heard that women shouldn't apply for some of those because they're viewed as special grants and you're not equal.

Caroline Wardle: Actually, I think we have some senior women in the audience here who have received such grants and could probably answer that better than I. You're referring to existing programs such as POWRE, which is the Professional Opportunities for Women in Research and Education. And prior to that there was visiting professorships for women and research planning grants for women. So there is a very small number, by the way, for women and it really is institution dependent. I have heard women in major research universities say it is not well regarded. And, of course, they vehemently disagree it should be and we can give you some ammunition that will help you educate like the department chair. We can give you the funding rates and you'll find out it's very tight. I don't think it's a 30% funding rate, I think it's way down. So it's very competitive. But I've had other -- I've had women at MIT say it was wonderful having the POWRE grant because they could spend it on anything they wanted. And she was on group grants where there was a lot of money but she didn't have a grant that she could spend for anything she wanted whether it was supporting a graduate student or buying equipment. So she found it wonderful. So it really does depend. And I don't know if we have any about those senior women here. It's better to go to the source, I think. There's some very well known women like Nancy Levison, people like that who had visiting professorships for women.

Audience: I haven't gotten one of those but let me say that -- let me say that I think from the PIs point of view, you know, money is money and if money helps you further your research career, that is a good thing. The thing that I think might be interesting, and maybe Janie can answer this in the tenure session, is does it make a difference to your tenure committee if the source of your funding has been specialty grants, a grant for women and minorities versus a grant that's for all technical people and a certain program or something like that. I think that's where it might make a difference but in terms of the PI, if you can get more resources to do your research, I would think go for it.

Caroline Wardle: I would say take the money and run. Yes. Very often the programs for women are to do something that you cannot get funded under a regular research program. They're different. And I would use those as a jumpstart to get you in and then absolutely aim for the regular research programs. So they are usually different. Ann, do you want to speak on some of this?

Ann: I just want to speak to that because I got one of the visiting professorships for women grant and I used it to augment sabbatical funds and visit the University of Washington, and I thought that the University of Washington really took it seriously that I had this and were really impressed that I was able to get it and the fact that it was a women's program didn't seem an issue at all. And it made a big difference because I had my own office. I would have had to share an office if I didn't have that grant. I was bringing in money to the University of Washington. I was supporting a student and so just my access to resources quite apart from the funding money of the grant itself completely changed as a result of getting this grant and I was able to work with the top professors in the country. So I put in a plug for these grants.

Audience: Actually, I was going to ask a question but I'll address that also. I'm from a minority institution, University of Texas at El Paso. We have been very successful with a minority grant which is to develop a model to bring undergraduates into research and this is directed at minorities and women. But even on our own institution, that kind of grant is looked at with less respect than another grant. And that's a shame.

Caroline Wardle: But I think it's changing. Universities now are becoming much more aware that they have to put more effort and focus on teaching. There's been a lot of pressure not meeting from parents but from legislature so we've really started to see a change happening and I think there's a lot of concern about education and I think bringing women and minorities in is also a part of that, a broader issue of diversity. So it is changing. It just changes slowly. But I definitely -- I have people coming in now that they're bringing their presidents along with them who are backing them for all of this and universities are starting to change the reward system. That's always the question I asked. Have you changed the reward system? Are you giving tenure to people who are superb teachers and have done some creative things? Are you promoting them? That's the key. It is starting to change.

Audience: And I do agree with you on that because our Provost just recently sent a memo indicating that the tenure decision process was being changed. So there is hope. But my question is on unsolicited proposals. I've heard actually different opinions with respect to NSF as to whether it is a good idea or not to submit an unsolicited proposal. So I was wondering if you could address that.

Caroline Wardle: At NSF, and what I have seen personally, is a fairly small number of solicited proposals. Our programs do have particular areas. I'll ask Frederica to comment. I think you're going to find they're very discipline dependent. I have seen big, very big, infrastructure grants coming unsolicited. We also, by the way, do have special projects. Where it's been used in my directorate is if you want to do something that doesn't fit under the existing programs, then it becomes a special project. But my advice would be always talk to the program officer that's closest to the area you're interested in and ask their advice. Frederica, what is your experience?

Frederica Darema: I want to corroborate that. That is certainly, if you submit and solicit a proposal you should speak maybe - to more than one program manager or director, to whom to send (the proposal) to, and typically people are very good in trying to find who are the program directors that can sponsor that proposal. But also it helps if you have done some of the homework yourselves by contacting the individual program managers and make them aware so when one of them receives your proposal, there is a little bit of awareness from their part about these proposals. So, yes, we are doing a lot of effort in addressing the proposals that are in between the different programs.

Audience: Hi. I'm from Rice University. I've heard that if you apply for an NSF grant, it is much more difficult to get money to support post doctoral salaries as opposed to graduate student salaries. Would you comment on the kinds of resources that your respective agencies are more or less willing to get funding for?

Caroline Wardle: All right. Should we let Jim have the start of this since he's been quiet for a while?

Jim Cassatt: No problem. There are -- we actually have more problems with graduate student salaries than we have with post doctoral salaries and there's actually something called reasonableness of cost. That is, we actually have dollar limits on what we will pay for the total educational cost of a graduate student on a grant. What was happening was we were seeing graduate student salaries of $16,000 quite reasonable. On top of that, $20,000 tuition, which gives you $36,000 versus a first year post doctorate for $28,000. People look at this and the GSA said, that's not reasonable and they were right so I think we actually have limits of what we will pay for the total cost of education of a graduate student. But really what a study section looks at when they look at the budget is how many FTEs are required to do this work and we certainly have no problems funding graduate students, post doctorates, and technicians.

Caroline Wardle: Frederica, do you want to say something? Do you want to address the NSF?

Frederica Darema: From DARPA's point of view, certainly funded contracts is the work to be done, there is no preference or negativism towards either post docs or graduate students. It is considerate part of the proposal whether it has a good management plan of what the work is with respect to the individual contributors to the project. But there is no specific emphasis -- because it's not a strong mission in DARPA to support graduate students as it is at NSF. But certainly DARPA recognizes the need of fostering the new researchers for the future and therefore graduate students are supported by DARPA.

Caroline Wardle: At NSF there is a lot of support for graduate students on the research grants. It is included as part of the budget and that's why we fund the graduate students. And that's very important to NSF.

Post docs. Well, you have hit a hot button here. Post docs are very controversial, probably in the external community as well as inside. The opinions are split in the CISE directorate. Now we're talking about computer science. If you talk about chemistry, mathematics, post docs are a normal course of your career path there in academia. In computer science they have never been a normal part in the same way. Very controversial. Many program directors believe there should be no post docs in computer science and some think there should be. Now we do have a program, we have a small program which does fund post docs in computational science and post docs in experimental systems and they are very specific and it's -- to take a post doc who has been trained in some computer science area and to retrain them in those areas, in computational science or experimental science. There's been lots of discussion over the all the years I've been at NSF. Should we have a post doc program? Should we not? It's still controversial just as it was nine years ago. So I think I would agree with your statement that if you put in a research proposal, funding to support a graduate student, that will be viewed as just part of a normal research proposal. A post doc would have much more difficulty unless it was very, very strongly justified. And it might depend on the program director. The program director's priorities might be not to support post docs. They would rather support another junior faculty member that's got on a tenure track line. So I think my suggestion there is, talk to the program director again and see if they would support a well reviewed proposal with a post doc and follow their advice.

Audience: Could I extend that question to addressing supporting programming, support programmers?

Caroline Wardle: That's harder at NSF depending on the projects. That's more of infrastructure so we have a lot of big equipment infrastructure programs where you can buy large equipment. Typically a department will come in or a big research group and we will allow technical staff on most of those which means the systems level or the network engineers but, typically, not the programming. Not the straight user and application programming. It's just that we're trying to leverage the money to the best possible way so we want to support the research rather than the applications developments later.

Jim Cassatt: NIH, again, doesn't care. The problem is -- I mean we have a number of projects with the primary emphasis not computer science but do computational something or other. And the program may be in the central part of that project and NIH will support it. The problem comes in the cost to programmers. They are a lot more expensive than biologists and the study sections look at this and start scratching their heads a lot.

Frederica Darema: So the emphasis, in a sense, reflects the missions of the agencies. At NSF the mission is generating research so, therefore, as Caroline said, is looked down that if you want to develop a piece of software you use your graduate students. At DARPA the view is different because you are supposed to develop products and certainly that is not looked as negative.

Caroline Wardle: I think we've just got time for one more question and one more minute left.

Audience: I'm from the University of Wisconsin. A quick comment and then a quick question. I had the good fortune of reviewing for a career and for power, successively, one fall and one spring, and the POWRE proposals were much more competitive than the career proposals. Much more difficult. I would encourage you, any of you that have a POWRE to find out what the funding rate was whenever you had it because --

Caroline Wardle: Does everybody know these acronyms? The CAREER is the five year program for new investigators. You have to be within, I think, four years of receiving your Ph.D. It's for men and women. That's an NSF wide program and CISE puts in lots of money into that. POWRE is the program for women and it could be junior or senior women.

Frederica Darema: Right. And that is reflected, in a sense, in the higher quality of proposals because when you have more senior people in the pool, certainly there is more experience brought into the proposal.

Audience: I did have one question. I am from, as a small number of people here, a primarily teaching institution where research is not the primary emphasis and outside of EHR, that sort of educational wing of NSF, are there any funding opportunities at all that are realistic to apply for? Particularly things like RUI? I've heard stories about research at undergraduate institutions. They have a program that it's more or less just a way, a vehicle for projects to add undergraduates onto their larger projects.

Caroline Wardle: Oh, no, RUI, Research in Undergraduate Institutions, is a program whereby you apply for faculty research, individual faculty research. You can apply for equipment but you have to be from the PUI, a Primarily Undergraduate Institute, and there's rules on that. These funny acronyms. I think it's a while since I did a RUI award but you have to fill in a special form that says you are a PUI and that means that you're -- like the liberal arts colleges, the comprehensive -- the Bachelor's and Master's degrees and small doctoral institutions are sometimes allowed to apply, if they don't' give more than 16 doctorate degrees per year, they're a PUI. And so this is a way to let people from smaller institutions come in with research proposals. I'm not answering research rather than education. And so you have to have an extra section in your proposal that is not in a regular proposal which is how is the work you're going to do effect the teaching at your university? So it's looking for you to take your research and bring it into the curriculum. So the RUI program is a set aside. We have to put a certain amount of money to proposals that come into the RUI program.

As mentioned, EHR is the educational directorate. That is a huge directorate with lots and lots of money that funds primarily educational proposals. Some of them are research and education but most of them are education applications. But each research directorate has a little bit of education. My directorate has one program called educational infrastructure which is a program that provides about $400,000 a year over three years to fund innovative educational activities. We jointly run a program with the engineering directorate, CRDC, and that's a combined research and educational program. Education is an area, by the way, that we will also entertain unsolicited proposals and we call them special projects. So we've had interesting things come in to do with education. They clearly don't fit any research program but they've come into special projects and we have funded some of those.

Again, the best thing to do is to ask the program director that you know, anybody you know, how would I send in an educational proposal. And depending on what you're suggesting we might suggest one of the educational directorate programs. We might say, that sounds interesting. Send us in a proposal and we'll deal with it as special projects.

Jim Cassatt: NIH has an area grant program specifically for people from smaller institutions. The grants are three years, $75,000 for three years; that's $25,000 a year but it is limited to smaller institutions for research at those institutions.

Frederica Darema: For DARPA, of course, it does not have this kind of specific programs, but the equivalent of that are the SBIRs which give about $100,000 for the first year and if the project is successful (and it's reviewed after the first year), then you get up to $350,000 for the subsequent two years. That's per year, the $350,000. So that's a very good way for enabling small businesses that want to do some research to also have this capability.

Caroline Wardle: I think we're out of time. Thank you so much for coming and please enjoy the rest of the workshops.