Aircraft Survivability is published quarterly by the Joint Aeronautical Commanders' Group, Joint Technical Coordinating Group on Aircraft Survivability (JTCG/AS).
Editor:
LTC John N. Lawless, Jr.
Naval Air Systems Command
Code AIR-4.1.8 (JTCG/AS)
1421 Jefferson Davis Highway
Arlington, VA 22243-5120
Phone:
(703) 325-0165, DSN 221-0165
Views and comments addressed to the Editor at the above address are welcome.
Mailing list:
Direct mailing list additions/deletions/changes to:
WL/FIVS/SURVIAC
Building 45
2130 Eighth Street, Suite 1
Wright-Patterson AFB, OH
45433-7542
Attention: Ken Applin
(513) 255-4840, DSN 785-4840
Live Missile Firings conducted by the Air Force at Eglin AFB in August 1994, providing surface-to-air missile (SAM) performance data in support of the SMART program's validation of SAM system computer models.
This issue of Aircraft Survivability focuses on survivability methodology. Within the DoD, a great deal of work in this field continues to be performed by numerous organizations. In the aircraft survivability field in particular, the use of modeling and simulation (M&S) has grown exponentially. As the high costs of weapon system development and testing come face to face with the harsh realities of declining R&D investment budgets and fewer personnel, M&S provides a cost effective means of analyzing system performance.
Both the complexity of models and our reliance on them continue to grow. The JTCG/AS Survivability Methodology subgroup plays a key role in coordinating the efforts of model developers and users. Dave Hall, the subgroup chairman, notes the increasing emphasis on verification, validation, accreditation (VV&A) and configuration management of M&S to enhance their credibility among senior acquisition decision makers. These and other challenges will continue to test the resourcefulness and ingenuity of the M&S community.
The Central Office is pleased to welcome our new Air Force civilian representative, Raymond R. Flores. Ray fills the slot vacated by John Over last April. In his previous assignment at Wright-Patterson Air Force Base, Ray was closely associated with the JTCG/AS as a section chief in the Survivability Enhancement Branch of Wright Laboratory's Flight Dynamics Directorate. He recently served as the Contracting Officer's Technical Representative (COTR) for the SURVIAC contract. We look forward to working with you, Ray.
We must also bid farewell to some longtime professional associates and personal friends. Air Force Principal Member Sol Metres retired in January after many years of dedicated service and has been replaced by Ralph Lauzze. Hal Reddick, John Graves, Dave Keyser, and Bob Yeager have also moved on to face new challenges. Our sincere thanks to all for their outstanding support of the JTCG/AS, and best wishes for much success and happiness in the future!
An important reminder: The 1995 Air Combat Survivability Symposium, sponsored by the JTCG/AS, will be held June 13-15, 1995 in Laurel, Maryland. The theme of our symposium, the first of its kind in over 7 years, is "Challenges in Air Combat Survivability: The Next 25 Years." Senior military and civilian representatives from both Government and industry will present their views on future threats and the challenges we face in defeating those threats. This issue of Aircraft Survivability provides more information about the symposium. We look forward to seeing you there.
Finally, I want to acknowledge a well-deserved honor recently bestowed on one of the true pioneers of the aircraft survivability discipline. Dr. Robert E. Ball, author of the landmark work, The Fundamentals of Aircraft Combat Survivability Analysis and Design, has been named a Distinguished Professor at the Naval Postgraduate School (NPS) in Monterey, California. This highly prestigious title has only been bestowed on three percent of the full-time tenured faculty at NPS. Congratulations, Dr. Ball, from all of us at the JTCG/AS!
John N. Lawless, Jr.
The JTCG/AS Central Office is pleased to welcome Mr. Raymond R. Flores. Mr. Flores comes to us from Wright Laboratory's Flight Dynamics Directorate at Wright-Patterson Air Force Base, Ohio. For the past 2 years he held a section chief position in the Survivability Enhancement Branch, where one of his responsibilities included managing the Air Force's Aircraft Survivability Research Facility. From 1987 to 1993, Mr. Flores was a senior project engineer assigned to Wright Laboratory's Advanced Structures Advanced Development Programs Branch, where he managed several multi-million dollar efforts aimed at developing promising structural concepts. He began his Government career in 1981 as a flight test engineer at the Air Force Flight Test Center at Edwards Air Force Base, California, where he was assigned to the B-1B Combined Test Force as lead airframe systems engineer.
A Registered Professional Engineer since 1984, Mr. Flores holds a B.S. in Mechanical Engineering from the University of Akron, an M.S. in Mechanical Engineering from the University of Michigan, and an M.S. in Aerospace Engineering from the University of Dayton. Mr. Flores was selected as the 1993 USAF Civilian Engineer of the Year by the National Society of Professional Engineers and was subsequently chosen as one of the Top Ten Federal Engineers of the Year. He also earned the designation of Certified Manager in 1993 from the Institute of Certified Professional Managers. His outside involvement's include running, motorcycling, sports officiating, and trying to keep up with son, Adam (11) and daughters, Sarah (8) and Emily (2). He and his wife, Marianne, reside in Alexandria, Virginia.
Mr. Sol Metres, Deputy Director of Wright Laboratory's Flight Dynamics Directorate and JTCG/AS Air Force Principal Member, retired on January 3, 1995. Mr. Metres has been instrumental in the research and development of flight vehicles and subsystems for aeronautical and space applications for nearly 35 years. His direct involvement with aircraft survivability began in 1975 with the assignment of survivability and vulnerability work to the Flight Dynamics Laboratory's Vehicle Subsystem Division. Since 1985, Mr. Metres was the Air Force Principal Member and he served as Chairman of the JTCG/AS Principal Members Steering Group (PMSG) from August 1988 through September 1991. His contributions to the survivability community in general and the JTCG/AS in particular are too numerous to list. Our best wishes to Sol as he faces the brave new world of retirement!
LT Gen Richard M. Scofield, Commander of the Air Force Aeronautical Systems Center, named Mr. Ralph Lauzze to replace Mr. Metres as the new Air Force Principal Member. Mr. Lauzze has long been associated with the JTCG/AS in his capacity of Joint Test Director of the Joint Live Fire (Aircraft) Program. In his letter dated 17 February 1995, LT Gen Scofield states that "Mr. Lauzze will represent the Air Force in ensuring that the JTCG/AS program continues to be relevant to Air Force survivability/vulnerability needs... I consider this to be a key assignment because as resources diminish, interservice cooperation is becoming even more important. The Air Force Principal must work with the other service principals and OSD to ensure that the JTCG/AS maintains a technically relevant, viable, balanced program, with payoff to all three services." Congratulations to Mr. Lauzze on this richly deserved honor, and we look forward to continuing to work with him in this new capacity.
Several other key personnel have recently left the JTCG/AS. Mr. Hal Reddick, Chief of the Safety & Survivability Division at the U.S. Army Aviation Applied Technology Directorate (AATD), Fort Eustis, Virginia, retired on April 3, 1995. Mr. Reddick has long been an active supporter of the JTCG/AS, serving most recently as Army Co-chairman of the Survivability Methodology Subgroup. His guidance, leadership, and broad understanding of the survivability needs of Army aviation will be sorely missed. Mr. Reddick's replacement has not yet been named.
Three committee co-chairs have changed within the Vulnerability Reduction Subgroup. From the Armor Committee, Mr. John Graves of the Army Research Laboratory in Watertown, Massachusetts, has left Government service. His replacement is Mr. Roy Burrows, head of the Flight Safety & Crashworthiness Section, Safety & Survivability Division at AATD, Ft. Eustis. Dr. David Keyser of the Naval Air Warfare Center in Warminster, Pennsylvania has stepped down as Flight Controls Committee chair due to a job change. Dr. Keyser is being replaced by Mr. Marc Steinberg, also of Warminster, who assumes the duties of Navy co-chair. The new Flight Controls Committee chairman is Mr. Bruce Clough of Wright Laboratory at Wright-Patterson AFB, Ohio. Mr. Clough replaced Mr. Bob Yeager of Wright Laboratory, who recently retired.
Development of the Advanced Joint Effectiveness Model (AJEM) is getting underway. This model is expected to replace a large number of existing vulnerability, lethality, and endgame models. As part of this effort, the JTCG/AS and the model developers are looking for broad Government and industry participation to ensure that the model meets or exceeds requirements for applicable types of analyses. Anyone interested in participating in this effort is asked to call LCDR David Hattery at the JTCG/AS Central Office, (703) 325-0165 or fax (703) 325-0178.
The Joint Technical Coordinating Group on Aircraft Survivability (JTCG/AS) was established shortly after the Vietnam conflict as a result of the large number of U.S. aircraft destroyed by enemy fire. When these aircraft were originally developed, survivability design requirements did not exist. As a result, one of the primary goals of the JTCG/AS was to establish survivability as a design discipline.
Over the years, Professor Robert E. "Bob" Ball has been a leader in helping the JTCG/AS reach this goal. In recognition of his many invaluable contributions in this area, Professor Ball was recently selected to the highly prestigious position of Distinguished Professor at the Naval Postgraduate School (NPS) in Monterey, California. Out of a staff of approximately 225 full-time, tenure track faculty members at NPS, Ball is one of only seven to be honored with this title.
Dr. Robert Ball receives Distinguished Professor award from NPS Superintendent, Rear Adm. Thomas A. Mercer
A highly accomplished and respected survivability educator, Professor Ball is the only known individual in the world who teaches all aspects of survivability. He first introduced a graduate level aircraft combat survivability course at NPS in 1977. At that time, survivability was not recognized as a formal design discipline, and no educational program for survivability analysis and design existed. Since then, Professor Ball has taught that course - as well as a shortened version sponsored by the Naval Air Systems Command (NAVAIR) and the JTCG/AS - to thousands of students, including Department of Defense (DoD) and industry engineers and managers. He also has developed NPS courses in air defense lethality and surface ship survivability and has conducted short courses for NATO and the governments of Canada and Greece.
"He is absolutely the best educator I have ever known, and we are lucky to have him in the survivability area."
Professor Ball, who wrote the only existing comprehensive and authoritative survivability textbook, is working on the second edition, as well as a second volume covering the survivability aspects of directed energy and unconventional weapons, for the JTCG/AS. He is also working with another professor on a textbook on surface ship survivability.
Professor Ball has organized and conducted several JTCG/AS survivability symposia and workshops, and he served as editor of the JTCG/AS Survivability Newsletter for several years. He is also a member of the steering committees for the DoD Survivability/Vulnerability Information and Analysis Center (SURVIAC) and the Air Warfare Section of the American Defense Preparedness Association (ADPA) Combat Survivability Division. In 1989, he established the Survivability Technical Committee of the American Institute of Aeronautics and Astronautics (AIAA). In addition, Professor Ball wrote the first article on military and civilian aircraft survivability for the 1994 McGraw-Hill Yearbook of Science and Technology.
"Survivability," says Ball, "is a science whose time has come. The design of every military aircraft in development today and in the future will include many survivability enhancement features."
Being named a Distinguished Professor means that Professor Ball is recognized as a "senior statesman" among his colleagues. His career has been distinguished by continued service to the NPS and the faculty. He has enhanced the stature of the school and made scholarly contributions that have helped establish survivability as a design discipline. The award, which entitles Professor Ball to the honorary address, "Distinguished Professor," consists of a silver medallion on a neck ribbon.
 |
Mr. David H. Hall is the Chief Analyst for the Survivability and Lethality Division of Naval Air Warfare Center, China Lake. He also serves as the Susceptibility Model Assessment and Range Test (SMART) Project Manager. After graduating from California State University with a B.S. and an M.A. in Mathematics, Mr. Hall started with the Naval Weapons Center, Corona Laboratories in 1968. He has been at the China Lake facility since his transfer in 1970. Over the years, Mr. Hall has acquired a wealth of experience in such areas as fuze and warhead design analysis, weapons systems requirements definition, mission effectiveness analysis, and survivability analysis. Mr. Hall can be reached at (619) 927-1297, DSN 469-1297. |
Q: What has been your affiliation with the JTCG/AS over the years?
A: I was first involved in the JTCG/AS soon after I transferred into the Survivability and Lethality Division here at China Lake back in 1976. I recall going to a JTCG/AS "Vulnerability Analysis" Subgroup meeting at Wright-Patterson Air Force Base sometime around 1978, when Jerry Wallick was chairman. I was involved on and off after that until about 1988, when John Morrow transferred to another job at China Lake and recommended me as chairman of the newly reorganized Methodology Subgroup. Since that time I've been actively involved in the JTCG/AS, not only with the Methodology Subgroup, but also with the Susceptibility Model Assessment and Range Test (SMART) project and the SURVIAC Technical Coordinating Group.
Q: What changes have you seen in your organization during this time? How is the Navy organizational consolidation going to affect your organization?
A: When I started working for the Naval Weapons Center in 1968, it was a fiercely independent RDT&E facility with a reputation for developing and constantly improving quality weapons systems for the fleet. Over the years since then, I believe due both to bureaucratic and downsizing pressures, we have seen some erosion of the Navy's capability to produce cost-effective weapon systems. Much of what used to be accomplished "in-house" at China Lake and at other laboratories is now done on contract, with the resulting delays and higher costs associated with the procurement process. What used to take 3-5 years to develop now takes 10-15. There are still pockets of technical work that are accomplished at the Center, one of which, thankfully, is survivability RDT&E; we have been fortunate to be able to maintain an effective workforce and accomplish significant technical work in-house.
I should say, rather, that our "competency" has been fortunate to maintain this in-house expertise. The biggest change locally in recent years has been the realignment of the Naval Air Systems "team" into a competency-aligned organization, the principal goal of which is to develop a seamless organization across all the sites dealing with Naval Air Systems, including China Lake, Pt. Mugu, Pax River, Warminster, NAVAIR, and Lakehurst. That is the basis for the competency alignment versus a geographical alignment. Once again, our survivability organization is somewhat fortunate since we already were "competency aligned" around the survivability discipline. The primary difference for us with this new organization is that some elements of survivability expertise at other sites are being drawn into our organization, where before they were more or less competing for project funding. I think, in the long run, this new organization could be to our benefit, although it is causing considerable consternation among the other "competencies" that were not previously organized around a particular discipline.
Q: What are the top challenges your organization faces over the next 5 years? How will you meet them?
A: The general budgetary draw-down is affecting us much like everyone else: as much or more work to do with fewer and fewer people. We just lost some experienced people in this last round of early retirement incentives, and that trend will continue. So our biggest challenge is to maintain a capable workforce of recognized experts in survivability analysis, engineering, and testing. Another challenge we face is keeping up with advances in computer equipment and software - our current workstation network is 5 years old - as well as straining to accommodate our expanding workload. How will we solve these problems? We plan to bring in some experienced people and train new ones. And the computer situation requires additional funding.
Q: How does Naval Air Warfare Center (NAWC) fit into the overall Navy modeling community? Into the joint community?
A: It's probably easier to describe how NAWC fits into the joint M&S community than it is to describe the Navy relationships. NAWCWPNS China Lake has always been the Navy's lead laboratory for survivability. Consequently, the Weapons Division has taken the lead for the Navy in representing the JTCG/AS and other joint organizations in the survivability area. The Navy survivability M&S position has always been to make use of joint Service simulations before we develop Navy-specific ones. As a consequence of that philosophy, very few Navy-developed survivability M&S are in use at NAWC. The majority are either Air Force or Army M&S that have been adopted by the JTCG/AS Methodology Subgroup as the best capabilities available. Our goal is to use only joint methodologies, and preferably those in SURVIAC, for conducting Navy survivability analyses.
Within the Navy, a number of organizations are involved with M&S policy, procedures, and usage. OPNAV N-6 (formerly N-8) has been newly designated as the Navy M&S Office. NRL has the lead in 6.1, 6.2, and 6.3-funded programs for M&S technology development, but very little of that funding is allocated for survivability modeling. Most of the Navy facilities doing aircraft survivability M&S development or improvement are within the NAWC organization, either at China Lake, Pax River, or Warminster.
Q: In modeling and simulation, where do you see the emphasis being placed over the next 5 years?
A: I believe in the survivability area, and in M&S in general, the emphasis will be on model credibility. M&S are being used more and more to help design weapons systems, and unless we know where they can be used and where they can't, we are going to be doing a poor job of applying them to system design. There is a growing faction among analysts (and, shockingly, even among some system engineers) that says we don't need to do testing at all, that we can just simulate everything we need to answer design tradeoffs. But how can you do that if you don't know that your model gives you the right answers in all the areas where you need answers? You can't!
OSD is starting to ask hard questions at Defense Acquisition Board (DAB) reviews about model credibility: How well does your model match reality? For what conditions? How do you know that? And do you know which version of the model you ran? Unless we can answer those questions, we're not doing our job as analysts. So I think that emphasis must be placed on verification, validation, and accreditation (VV&A) and configuration management of the M&S that are used in support of the acquisition process. Configuration management is the long-term key to maintaining the "shelf life" of VV&A conducted on a model: we must transition knowledge about the model from version to version, or there is no long-term interest in doing VV&A. Consequently, the JTCG/AS Methodology Subgroup is going to start funding V&V documentation on the models being distributed by SURVIAC and participating more directly in SURVIAC model configuration management.
Q: Tell us about SMART, DMSO, DIS, EDSWG, FACTR, TILV, DIME, and J-MASS and their connection with the JTCG/AS.
A: Some of these acronyms represent projects that are directly supported by the Methodology Subgroup, and some represent related efforts. SMART is addressing the model credibility issue by developing a joint Service V&V process for survivability M&S. The Defense Modeling and Simulation Office (DMSO) is funding an effort to expand the SMART V&V process to a larger class of M&S, including mission-level M&S and Distributed Interactive Simulation (DIS)-related modeling activities. The Electronic Defense Simulations Working Group (EDSWG) is part of an Air Force-funded activity trying to standardize M&S of threat weapons systems within the Air Force; this group has started its effort by making extensive use of the SMART-generated modeling templates built on the ESAMS model.
The Future Air Combat Tool Requirements (FACTR) study is being conducted for the Methodology Subgroup and for the Air Force M&S office (AF/XOM) in an attempt to bring together the BRAWLER and AASPEM communities to develop a single set of requirements for air combat modeling. The goal is to use the JTCG/AS as a vehicle to agree on one fixed-wing air combat model that meets everyone's needs.
The Target Interaction Lethality/Vulnerability (TILV) committee is charged by the Joint Directors of Laboratories with coordinating [exploratory development-funded] vulnerability and lethality M&S developments across the Services. It has asked the JTCG/AS to participate in defining requirements for aircraft survivability-related vulnerability modeling.
Finally, the Digital Integrated Modeling Environment (DIME) is a JTCG/AS-funded project to improve on an Air Force-funded environment for running the SURVIAC models in a threat scenario. The Joint Modeling and Simulation System (J-MASS) is another Air Force-funded project (it has received joint funding in the past) to develop a standard modeling architecture for DoD. We have recently been asked to become more involved in the J-MASS user group to define requirements for J-MASS from an aircraft survivability analysis perspective.
What does all of this mean to the JTCG/AS? I think that the most important part of any new M&S development, or any new architecture development, is transition. Unless we can provide a painless transition for the analysts from the M&S they're using today to a new capability, they're likely to just modify a "tried and true" model to meet their needs and completely ignore (or actively vilify) any new development. And I don't believe model developers really understand that attitude. The people who develop models feel that once they've demonstrated a capability, everyone will beat a path to their door. But, better mousetrap or not, most people will continue to use the mousetrap they already have unless you can convince them it's to their benefit to buy a new one. I think the Methodology Subgroup can be the vehicle to provide that transition by working with the different groups involved to temper their natural enthusiasm for newer and better M&S with the cold, hard realities of user requirements. JTCG/AS represents survivability M&S users, and you won't find a better or broader representation of that community anywhere.
Q: What modeling shortfalls currently exist within the survivability community, and what is being done to address these areas?
A: The principal shortfall in survivability M&S capability is, in my opinion, the lack of credible countermeasures models. This is not to say that we cannot model electronic countermeasures (ECM) or infrared countermeasures (IRCM), but, for the most part, there is very little credibility in those models, especially in the radio frequency (RF) spectrum. The Digital Infrared Surface to Air Missile Simulation (DISAMS) models, for infrared (IR) systems, are the closest we come to "emulative level" models; that is, these models are built from engineering drawings of actual threat systems, and they have been proven to very accurately reflect IR countermeasures effects. We have made some inroads into RF countermeasures modeling through work funded by the SMART project, but that was a very limited demonstration of the approach to using test data to set model fidelity requirements. We need to expand on that effort, and that is being proposed by Dr. Marion Williams, the Chief Scientist at the Air Force Operational Test and Evaluation Center (AFOTEC) as a joint test project designed to support ECM model validation and improvement.
Another major shortfall is the lack of a standard methodology for evaluating the survivability of multiple airborne weapons systems in an integrated threat environment with multiple support assets. How can you trade off ECM capability with reduced signatures with improved performance and all the associated weight, cost, and performance impacts unless you can do so in the scenarios and environments in which you anticipate having to operate? The DIME project is our first attempt to address this issue within JTCG/AS, but it offers only a small part of the solution. The Methodology Integration Committee is looking at long-term solutions, possibly involving tools developed under J-MASS.
Q: What new models are likely to be developed or approved in the near future?
A: DIME will be the next thing to come out of the Methodology Subgroup and into SURVIAC later this year. The Army, with JTCG/AS support, is working on combining its two helicopter air combat models (HELIPAC and HAVDEM) into a single model (to be called TRACES), which will be distributed through SURVIAC sometime next year. The Advanced Joint Effectiveness Model (AJEM) development is just getting started by JTCG/AS and JTCG/ME to replace the current vulnerability and missile endgame codes (COVART, HEIVAM, HEVART, JSEM). If we can get sufficient funding and all agree on what the real requirements are, then AJEM could be the solution to many of the problems we currently have with vulnerability modeling. Longer-term efforts include the joint FACT fixed-wing air combat model initiative and J-MASS related issues.
Q: Which Navy programs do you think will have large analysis and modeling requirements in the future?
A: JAST is the primary target for much of the modeling effort in aircraft survivability, both for the Navy and the Air Force; other joint programs include JDAM and JSOW. Navy-only programs include the F/A-18E/F and cruise missile.
Q: What are your perspectives for the future of the survivability discipline as we face military draw-down? Has the survivability "war" been won?
A: I can only speak to analysis issues and not to the survivability discipline as a whole. From my perspective, the military draw-down means that more analytical results will be required than ever before with more credibility, since we can no longer afford much of the testing that would "normally" occur. And that's why we need to place so much emphasis on V&V (and configuration management of the resulting M&S and databases), so we can have confidence in the design tradeoffs and cost and operational effectiveness analysis (COEA) results that are generated with those M&S. In a way, the survivability "war" has been won, because survivability is recognized as an important design discipline within DoD. The war has not been won, however, in the sense that survivability design features are not automatically included in new designs; we still have to do battle with all the other disciplines for space, weight, and cost allocations as part of the overall systems engineering process. And that's how it should be.
SURVIAC has recently expanded its Internet capability and access for its users. As part of this effort, SURVIAC, along with the other DTIC IACs, has added a home page to the World-Wide Web (WWW) where users can find out how to get in touch with SURVIAC, find out what services SURVIAC provides, and sample some of SURVIAC's products. SURVIAC's home page on the WWW enables access to the SURVIAC Bulletin, the SURVIAC calendar of events, a list of SURVIAC products, and the JTCG/AS newsletter Aircraft Survivability.
The SURVIAC home page can be accessed by entering its URL (Universal Resource Locator):
http://surviac.flight.wpafb.af.mil/The DTIC IAC homepage can be reached by using the URL:
http://www.dtic.dla.mil/iacSURVIAC plans to continually update the information available at its web site, adding products and services.
People either love or hate computer models and simulations (M&S). This statement is borne out by the last JTCG/AS order-of-merit (OOM) ranking process. For those unfamiliar with the JTCG/AS OOM ranking process, proposed projects are rated by a number of different individuals associated with the JTCG/AS. The criteria are explicitly defined, as are the possible points for each area. The ratings are combined to yield a cumulative merit value for each project, and the projects are prioritized by that merit value into an OOM list of all projects. JTCG/AS funding is provided for projects that are highest on the OOM list and, hence, have the most merit.
If projects have an intrinsic merit, then in the very structured JTCG/AS OOM process, one would expect that individual projects would receive fairly consistent merit ratings from different raters. This case is largely true for both Susceptibility and Vulnerability Subgroup projects. For methodology projects, however, the ratings bestowed on a given project often span the entire rating scale.
There are several possible reasons for the wide variability in the ratings. The most likely reason is that, although the raters understand what the project purports to do, they have very different beliefs on the aircraft survivability value of the project. Most people agree on what vulnerability reduction brings to the aircraft survivability discipline, even if they believe that the money would be better spent on susceptibility reduction. This case does not seem to hold true with aircraft survivability M&S. The link between development of these M&S tools and increased survivability is much less clear, and individual beliefs on this vary tremendously.
To determine the value of M&S, one must first identify the impact of M&S on aircraft survivability. On the face of it, money spent on M&S is money that does not go toward hardware that will directly benefit the operational forces. At the other end of the spectrum is the belief that M&S will soon be used to completely design and test weapons systems before any hardware is built, thus reducing the time and cost of product development. The true impact lies somewhere in the middle, but where?
The fact remains that most acquisition programs owe at least part of their existence to M&S-based studies that demonstrated their value to the people with control of the money. To save money, program managers (PMs) are increasingly selling the merits of their programs with M&S-based information rather than with the results of more expensive hardware testing, which was predominantly used in the past.
Whether this change is for the better is still open to discussion. Many believe that M&S will never attain the level of capability needed to completely replace testing. The value of M&S in properly designed trade-off and comparison studies is less contentious. Even without a verdict on the proper uses for M&S, M&S are being used increasingly to justify equipment, enhance capabilities, and test equipment performance. Given this trend, some level of effort is warranted to improve the capabilities of aircraft survivability M&S tools.
It is true that an incredible amount of money can be spent on M&S development with virtually nothing to show for it. As the defense budget shrinks, many agencies are forced to reduce the number of projects they are funding. Most are opting to reduce spending on areas without a clear near-term payoff. M&S very easily fall into that category. This situation is a cause for concern, since many believe that M&S have extremely large long-term paybacks.
Determining the return on investment for M&S is extremely difficult. Recognizing this, the Defense Modeling and Simulation Office (DMSO) is asking for anecdotal information that quantifies the impact of M&S within the Department of Defense (DoD). DMSO will accept data expressed in dollars, readiness ratings, number of successful or productive missions, or anything else quantifiable. DMSO also is looking for data that specifically quantify the return on investment in M&S. Like the impact of M&S, return on investment can be shown in different ways, including time saved and dollar benefit. DMSO's purpose is to better understand the real-world impact of M&S, "either positive or negative."
The next challenge is finding a good home for M&S development. In these days of tight budgets, program offices are considered one of the best sources of funding. The trouble is that a program office is almost exclusively focused on the success of its program. This can present a conflict of interest when weighed against developing a universally acceptable M&S tool that can be used by others. In addition, developmental time for M&S can greatly exceed the time available to the program office. Thus, a program office typically prefers to take an existing application and make a few minor changes. An agency that has a long-range vision is required to invest in the development of these tools.
This state of affairs surfaced at a recent meeting of the Joint Directors of Laboratories (JDL) sub-subpanel on Target Interaction Lethality and Vulnerability (TILV). The panel has created a TILV Science and Technology (S&T) master plan that identifies Lethality and Vulnerability M&S developmental requirements and ongoing work. The aircraft survivability vulnerability M&S requirements are well represented in the draft of this document. Those requirements were collected at a workshop, held last April, attended by numerous JTCG/AS vulnerability methodology individuals. In addition, the Army's air vulnerability M&S requirements are being addressed by S&T funding that is going to the Army Research Laboratory (ARL). Conversely, for fixed-wing aviation, only the lethality concerns of weapons developers are being addressed. There appears to be an absence of Air Force and Navy S&T work to address aircraft vulnerability M&S requirements. While lethality M&S and vulnerability M&S differ primarily in the perspective, the priorities are often different. One case in point is the development of a fire and explosion model. The weapons designers are principally interested in kill modes that do not require detailed fire and explosion models. The survivability community, on the other hand, is very interested in fire and explosion since it is a leading cause of Blue aircraft losses.
The Services typically have relied on the JTCG/AS and the JTCG/ME (Munitions Effectiveness) to invest in long lead M&S tools. Insufficient funding for the two JTCGs has prevented major investments in enhanced M&S tools. Therefore, changes to M&S used by the aircraft survivability community have been modest incremental enhancements aimed at increasing the serviceability of M&S that have been around for a long time (the "legacy models"). The JTCG/AS philosophy is to provide seed funding for promising technologies, financially leverage projects with other agencies, and then completely transfer those projects to the Services within a few years. So, by comparison, JTCG/AS funding of M&S improvements takes on the character of infrastructure maintenance. Improvements to a legacy model often continue over decades. Although the Services typically look to the JTCG/AS for the best M&S, development of those M&S does not neatly fit into the JTCG/AS funding paradigm. This issue may contribute to the diverse ratings M&S projects receive in the OOM process.
For the JTCG/AS to accomplish its long-term mission of providing the best M&S tools to the users, we must either identify Service agencies that can provide the bulk of the funding necessary to develop the next generation of M&S tools or identify new sources of funding. One suggestion is a tax levied on M&S users. Regardless, practical applications of the developmental M&S tools will sometimes be elusive and far down the road. This has been a critical point of several M&S endeavors to enhance the state of the art. The researchers are quick to point out the great (potential) capabilities of their work. Analysts, on the other hand, are quick to point out that M&S do not satisfy current requirements. The JTCG/AS should serve to put both of those valid points into perspective and serve as an interface between today's customers and far-sighted M&S developers. This will ensure that the community obtains the best M&S and that those M&S satisfy actual aircraft survivability requirements.
One area affecting the potential return on investment in M&S is credibility. Lack of M&S credibility can severely constrain M&S uses - and hence limit the possible payoff from using M&S. Many people consider computer models a new plateau in the art of deception. Brian Arthur, an economist from Stanford University, captures a common perception by noting that, early in his career, he believed "by and large people who couldn't think analytically resorted to computer simulations...[and] that you could prove anything you wanted by tweaking the assumptions deep in your model." Computer M&S are permeating every discipline and are being accepted as necessary tools, but the use of those M&S must be watched carefully.
The key challenge for M&S developers is building model credibility. The talent of M&S developers is wasted unless the results are believed by the people who count. The value of M&S is tied to the credibility they have in the eyes of key decision makers. Therefore, it is in the interest of M&S providers to go out of their way to make their M&S as credible as possible, and the users of M&S will demand as much. It is also in the interest of hardware designers to support the M&S development since they will probably be used to justify the purchase of the hardware they design.
Appropriately, there has been a tremendous effort to establish processes that assist decision makers judge the strengths of arguments based on M&S. Two principal measures attempt to characterize a model's inherent goodness: verification and validation (V&V). Although these terms are defined in JCS, DoD and Service documents, many processes are used throughout DoD to verify and validate M&S. Since models are constantly changing, a model's V&V is never complete. It is therefore necessary to track and document the model's V&V status with each configuration change. The SMART project has defined a flexible, technically sound and affordable V&V process that has enhanced the technical merit of several susceptibility models. Currently, SMART is investigating how its process could benefit other users of M&S, including users outside the aircraft survivability discipline.
Accreditation has been glibly defined as "an official determination that the M&S is good enough for Government work." Good M&S are meaningless, however, if the application to which the M&S are applied is not appropriate. Most agree that the "Government work" in which the M&S will be used must be very specific for an accreditation decision to be meaningful. The SMART project has done much to bridge the gap between V&V and accreditation by designing V&V documents that support a technically sound accreditation decision.
Another aspect of accreditation is convincing others that the accreditation decision is sound. This is being addressed by the DMSO Simulation Endorsement Process (SEP). A key objective of the SEP is to obtain up-front Office of the Secretary of Defense (OSD) buy-in to the use of particular M&S to support high visibility programs. Thus, a program office can return for an OSD review after extensive M&S-based studies and discuss data and results rather than defend the choice of M&S. Success in doing this can save years of rework and cost escalation due to program delays.
Although the impetus to create the SEP originated with the Joint Aeronautical Commanders Group (JACG), the SEP has a far broader scope than aircraft survivability. Domain expert reviews of M&S choices and uses are needed to give the process a sound technical foundation. Unfortunately, not all M&S domains have a joint organization such as the JTCG/AS. This has impeded efforts to create a simple process structure that covers all uses of M&S. The Joint Advanced Strike Technology (JAST) program will test the SEP and determine if the benefits of this process outweigh the burdens. Even if a universal endorsement process with high level buy-in is determined to be unworkable, M&S domains with a joint user group such as the JTCG/AS can still realize a significant jump in M&S credibility by establishing a peer review process.
The future of M&S is linked to the notion of a favorable return on investment in M&S development. It may not be possible to obtain a consensus on this. Establishing the credibility of M&S-based studies is the key element in the return on investment equation. Although the SMART project has done much to codify a process that establishes and documents the technical goodness of M&S, this process alone is not sufficient. Rigid regulations will not prevent this and will add to program costs. The JTCG/AS Methodology Subgroup is a professional organization with a vested interest in the credibility of M&S-based studies. The members of this community must build a core of integrity and hold other members to these standards. In this way, M&S will achieve a level of credibility not otherwise possible.
This article describes how the Susceptibility Model Assessment and Range Test (SMART) Project's accreditation support process is being applied to real programs with real accreditation requirements across the Services, and what this expansion of SMART's capabilities means to the final phase of SMARTs overall mission.
SMART's mission is to develop, test, establish, and transition to DoD a proven and efficient tri-Service credibility assessment process for mature, joint-use aircraft survivability M&S. This process is very important because M&S are being used more and more to make system acquisition and testing decisions, which have multi-million (or billion) dollar impacts. Halfway through this penultimate year of the project, the situation is even more urgent. Both DoD and the Services agree that the use of M&S in acquisition and testing needs to be substantiated with objective evidence, that V&V to supply this evidence is needed, and that V&V needs to be done now. V&V can be approached in many ways; some of these ways have high value, some have low cost, and none have both. Our mission in the endgame of SMART is to demonstrate that it is possible to balance value and cost. We are supporting three major system development and acquisition programs - the Tomahawk Program, the Joint Advanced Strike Technology (JAST) Program, and the Ballistic Missile Defense Organization (BMDO) - that may help us demonstrate this point to the wider community.
Figure 1. Survivability Models Used with TLAM Planning System - Threat Assessment Module
SMART's work with the Tomahawk Program began about 9 months ago. The models contributing to the Tomahawk Land Attack Missile Planning System (TPS) (see Figure 1) were coming under increasing scrutiny because of increased DoD emphasis on credibility, especially VV&A. Our work on this program was the first opportunity to apply the lessons learned from our Accreditation Requirements Study (ARS) and V&V experience to a practical problem. SMART supplied the accreditation support and V&V expertise, while Tomahawk supplied the sensitivity analysis of the TPS required to generate acceptance criteria for its contributing models (ESAMS, ALARM, RADGUNS, and TRAP). The TPS sensitivity analyses provided an interesting means of testing the consistency of the predictions across a wide array of scenario input conditions, and the stability of the predictions across this spectrum of scenarios enhanced confidence in TPS outputs. The sensitivity analysis results set bounds for acceptable M&S input fidelity. Functional, operational and fidelity requirements were set (see "Accreditation of Survivability M&S" in this issue of Aircraft Survivability), and an accreditation plan for each model was developed. We are currently finalizing these plans for Tomahawk and hope to brief the results to the Cruise Missile Project Office chain of command this spring.
Our relationship with JAST came about as a serendipitous meeting of requirement and capability. While surveying the M&S community for V&V expertise, the JAST Program Office asked us to provide a summary of our V&V experience and a proposal for the V&V of four of the JAST models: ESAMS, RADGUNS, BRAWLER, and EADSIM. We knew how to pitch the first two models based on our work on them for almost 2 years. The scope of the required BRAWLER and EADSIM work was a cipher, however, so we decided to use our Accreditation Support Package (ASP) concept as a template for accreditation requirements. We identified the V&V information gaps for the models and, using SMART's history of V&V costs, projected what it would cost the JAST Program to fill in the gaps. In the process, we convinced ourselves (and JAST) that our prior work on ESAMS and RADGUNS would significantly reduce the cost to accredit these models and that our cost and schedule projections for EADSIM and BRAWLER were both feasible and accurate. As a result, JAST funded SMART to conduct Phases I and II of EADSIM and BRAWLER V&V, work that is scheduled to begin this spring.
Our relationship with BMDO developed through a fortuitous mutual interest in EADSIM with JAST. BMDO is using EADSIM as part of a Theater Missile Defense (TMD) Cost and Operational Effectiveness Analysis (COEA). After the M&S Directorate in BMDO was briefed about SMART, they agreed to add participatory funding to the V&V for EADSIM, sharing the cost with JAST. SMART is now supporting BMDO's TMD COEA directly and coordinating acceptance criteria development for EADSIM in conjunction with JAST. By doing this, we hope to eliminate duplicative V&V on EADSIM for the two applications. Preliminary investigations into EADSIM model management also proved encouraging. EADSIM development seems well-disciplined, and much of the information required to produce ASP-I (the Model Characterization Report) is already available. With this leg up on V&V requirements for EADSIM accreditation, SMART and BMDO are now focusing on the much more difficult Phase II assessment (Expert Review) and Phase III (Detailed V&V) requirements prioritization. Specific EADSIM V&V tasking is scheduled to begin in May, under SMART direction.
The EADSIM success story has one more chapter to it. In briefing SMART's V&V process and products to the DMSO, we learned that they were interested in our expertise as an example of enterprising V&V technology that could be useful to the wider DoD community. As part of its larger goals articulated in its "DoD M&S Master Plan," DMSO agreed to share the costs of the EADSIM effort to expand the scope of applicability of the SMART V&V process to mission-level models. We now have three sponsors funding the V&V of EADSIM, all coordinated and executed by SMART. Two major programs and the DMSO are receiving accreditation support products at much less cost than if each undertook efforts individually. We are pursuing the possibility of jointly executing the V&V of BRAWLER and SUPPRESSOR with similar participation. The age of cost-effective accreditation may be dawning.
The DMSO will also play a major role transferring a recognized accreditation support process and infrastructure to DoD. DMSO has agreed to let SMART assist with V&V process inputs for the Aggregate Level Simulation Protocol (ALSP) confederation of models. SMART will contribute to V&V process development for this distributed simulation capability using lessons learned from prior work on other types of M&S. In addition, the DMSO has funded SMART to develop an accreditation support services action plan for possible FY96 implementation. This action plan will identify what kind of accreditation support services need to be provided based on M&S community needs, and will correlate these requirements with existing capabilities and facilities across the Services. The plan will also identify accreditation support service gaps, identify potential candidates and implementation strategies for filling these gaps, and provide preliminary budget and schedule projections for providing accreditation support services across DoD. The plan will probably include a pilot program for FY96 implementation. In conjunction with the current focus within the DMSO central office on developing coherence among and between rival Service VV&A approaches, this task will place the DMSO at the forefront of M&S credibility technology development.
As SMART draws to a close, we feel like we have built a better mousetrap, but the world has not yet beaten a path to our door. Continuing efforts like those described above are our effort to beat a path to the M&S world's door. It is within these efforts that the greatest audience exists for the dissemination of SMART lessons learned into the wider M&S community, and we harbor high hopes for passing on some form of SMART's hard-won experience as a legacy. Look for more on the subject in a future issue of Aircraft Survivability.
In the summer '94 edition of Aircraft Survivability, we summarized SMART Project contributions to survivability modeling and simulation (M&S) in terms of the baseline credibility assessments we had produced for three of the five models in the SMART set (i.e., ESAMS, ALARM, and RADGUNS). These assessments consist of a series of Accreditation Support Packages (ASPs) that contain the results of the following verification and validation (V&V) activities: model characterization, expert review, code verification, and validation with test data. (The Model Characterization Report [ASP-I] for the fourth SMART model [the Trajectory Analysis Program, or TRAP], should be available in June, with other ASPs following over the next 6 months; V&V work on SMART's last model, BRAWLER, will begin in FY96.)
This article describes SMART's approach to accreditation support for M&S based on over 3 years of experience in V&V. SMART has developed a five-step accreditation support process into which its three-phase accreditation support products fit. This process was developed on the basis of a merger between the Accreditation Requirements Study (ARS) conducted by SMART in FY94 and "applied common sense." (Copies of the ARS are available from the SMART Project Office; call Jennifer Wood at (619) 939-3680.) The ARS identified tri-Service information requirements that supported accreditation based on a review of emerging policies, procedures, and guidelines and ongoing accreditation efforts. The correlation and consolidation of these information requirements across the Services suggested to us the three-phase V&V process and products summarized in Figure 1. It was the review of actual ongoing accreditation work, however, that suggested to us a generic and practical accreditation support process that could guide M&S accreditation planning and execution for actual applications. The five key steps to developing a rational basis for M&S accreditation are discussed below.
Figure 1. SMART V&V Process and Products
Step One: In-depth analysis of the application. Before any decisions are made about applying M&S to a problem, the problem must be defined and articulated well enough to permit a specification of where and how M&S will contribute to solving the problem. An ill-defined problem is the most common reason for failure to integrate M&S into an analysis in a credible way. The problem analysis itself consists of four tasks: (1) identifying study objectives, (2) developing an agreed-upon set of measures of effectiveness (MOEs) by which each study objective will be addressed and resolved, (3) identifying which study objectives will be addressed or resolved using M&S, and (4) identifying the predictive capabilities that M&S must have to be of value to the study MOEs (i.e., functional requirements). Mapping of study objectives, MOEs, and M&S functional requirements is the most important aspect of the accreditation support process because it forms a template for the entire study plan.
Step Two: Development of M&S acceptance criteria. After defining what functions M&S will be required to perform in the study, how well those M&S must perform those functions must be determined. The two types of acceptance criteria developed in this step are operational requirements and fidelity requirements. Operational requirements are non-analytical requirements, in that they do not contribute to study MOEs directly. Instead, these requirements define such elements as hardware and software compatibility requirements (e.g., the M&S must run on a certain type of workstation under a certain operating system), pre- and post-processing requirements for M&S data (e.g., M&S inputs or outputs must be converted to special file formats), and operations and training support requirements (e.g., M&S cannot have license agreements or operator training requirements because there is no budget for such items or no time for training). Fidelity requirements list how well each required M&S function must correlate to real world data for M&S outputs to be considered acceptable for the purpose at hand. (This does not imply an absolute standard of fidelity for all applications but, rather, a level of fidelity considered "good enough" for the purpose at hand.) This list normally requires developing a notional "error budget," whereby variations in M&S outputs can be related to variations in study results via the MOEs. Fidelity requirements are easily specified when the standardized V&V information elements developed by SMART for incorporation into its ASPs are used. Although these elements provide a convenient checklist for fidelity requirements, the amount of V&V required to establish credibility for a particular application still depends on a clear definition of the study problem.
Step Three: Comparison of acceptance criteria with M&S characteristics and selection of M&S. The functional, operational, and fidelity requirements developed in the first two steps constitute the basic checklists of acceptance criteria against which M&S characteristics must be compared. Information on M&S capabilities is compiled from documentation, product literature, and other sources, and then compared to the functional requirements list to determine if any of the required functions are not modeled. Information on M&S operational characteristics (e.g., memory required, programming language, length of time to run a typical case, hardware and operating system requirements, training and maintenance requirements) is obtained from similar sources and compared to the operational requirements list to determine if additional resources will be required to maintain and operate candidate M&S during the study. Finally, the fidelity requirements list is compared to the verification, validation, and accreditation (VV&A) histories and current results of the candidate M&S to determine the applicability of prior V&V and to identify requirements for additional V&V to address the current problem. Next, a final selection of M&S for the study problem is made on the basis of these checklists. Acceptance criteria gaps that are identified are used to develop an accreditation plan that prioritizes each gap and describes how each will be addressed. Cost and schedule projections are made, and risk assessment and mitigation strategies are developed. The plan is briefed to the study agent (the person who pays the bills and determines the frequency and format of progress reports) for approval.
Step Four: Execution of the approved accreditation plan to fill the identified M&S acceptance criteria gaps. If the plan includes requirements for additional V&V, the SMART process (or any other acceptable process) can be used to meet the fidelity requirements and report the results. If the plan includes requirements for model development, V&V should be performed concurrently. When sufficient evidence for M&S accreditation has been accumulated, an accreditation report is written, summarizing how the gaps in M&S acceptance criteria have been addressed and/or resolved, and providing a risk assessment of any remaining gaps.
Step Five: Recommendation to either accredit or reject the M&S. The accreditation report concludes with a recommendation to either accredit the M&S for the application at hand or reject it on the basis of failure to meet critical acceptance criteria.
This five-step process provides an audit trail of well-defined study objectives, M&S acceptance criteria, and study results, including decisions that substantiate the use and acceptance of M&S results in the study problem. The advantages of this approach to the individual study agent, who must wrestle with an increasingly skeptical decision-making community, are clear. Less obvious, however, is the benefit to the wider user community of the accumulation of M&S credibility in accordance with this process.
As individual models are used again and again in support of different applications, substantive, objective evidence of their credibility grows. Although each user (or group of users) contributes only those aspects of credibility applicable to the current problem, an objective body of evidence grows over time. SMART's standardized V&V process, coupled with its standard V&V reporting format (the three-volume ASP format), provides a convenient way to accumulate evidence of M&S credibility. The steady accumulation of such evidence reduces the cost of subsequent accreditation for individual M&S, because accreditation builds on prior V&V rather than on independent (and mostly duplicative) efforts. Seen in this light, SMART's accreditation support approach provides the DoD community with the means to obtain lower-cost, high-value accreditation.
SMART has always been charged with the task of transferring its V&V process and products, as well as its accreditation support experience, to the wider DoD community. Ongoing efforts to apply SMART's expertise to current problems, and to continue these accreditation support services beyond the scheduled end of the project (currently FY96) are discussed in "SMART Comes of Age" in this issue of Aircraft Survivability.
Although the name of the Digital Model Working Group has changed to the Electronic Defense Simulations Working Group (EDSWG), our original guidance remains the same. This guidance includes defining requirements, pursuing joint capability, saving money, eliminating model duplication, making modeling and simulation (M&S) measures and results square with those of test and evaluation (T&E), and supporting development and demonstration of the Joint Modeling and Simulation System (J-MASS).
Model development is big business. Factors include the technical complexity of modeling surface-to-air missile engagements in an electronic countermeasures (ECM) environment, the variety of efforts underway, and the natural bias of competing organizations. In addition, J-MASS has not delivered on promised capability, and M&S budgets are under scrutiny. One thing is clear: for the EDSWG to be successful, business as usual is not satisfactory.
The EDSWG has achieved some impressive results. To evaluate the current state of engagement-level modeling and draw a roadmap for future development, we undertook the formidable task of defining requirements for the broadest possible list of users. We invited Army, Air Force, and Navy analysts, as well as Defense Intelligence Agency experts and several industry representatives, to participate. As a result, we developed a document containing an extensive description of the capabilities needed to model the effects of various ECM techniques on specific radar-directed threat missile systems. We did not make up our own list of favorite threats but looked at those identified as critical to planned T&E of the B-1B Defensive Systems Upgrade Program. We found general agreement among experts on which techniques are most effective against each threat and then matched several candidate techniques with each missile system. We considered factors such as antenna pattern and effective radiated power and made the requirements template independent of aircraft type to expand its applicability. We made no attempt to categorize requirements as emulative, analytic, or dynamic - we leave that for symposia discussion. Instead, we focused on the fidelity necessary to support electronic combat testing.
Technical capability of simulations is not the only issue. We included such things as usability, supportability, documentation, and portability across different computer platforms. Because navigating through the resulting mountain of information could discourage the most determined user, we utilized hypertext. The entire template is designed in a hierarchical structure to make finding information an easy point-and-click operation. It also allows for structured growth. Assuming the EDSWG's modeling requirements will be incorporated into policy by our Headquarters USAF sponsors, (the Deputy Chief of Staff, Test & Evaluation, and the Directorate of Modeling, Simulation and Analysis), other threats will have to be addressed. Bearing that in mind, we are turning to infrared guidance systems to repeat the process.
Finally, the EDSWG effort will be adopted as the initial set of user requirements for J-MASS. EDSWG is proud of its accomplishments and will continue to work on improving M&S efforts.
In the Fall 1993 issue of Aircraft Survivability, Mr. Dave Hall, chairman of the JTCG/AS Methodology Subgroup, introduced the vision statement developed by the Methodology Integration Committee (MIC) as the guiding force for subgroup activities:
"...to establish an accepted joint Service methodology for conducting air weapon system survivability analysis using a flexible and efficient computational environment based on a set of credible modeling components."
In the Summer 1994 issuer, the Methodology Integration Committee (MIC) chairman outlined how the Digital Integrated Modeling Environment (DIME) project was addressing the "flexible and efficient computational environment" part of the vision statement and how it could best be realized.
Now, in a project entitled VIsion Statement INtegration Evaluation (VISINE), the Methodology Subgroup is taking a closer look at how JTCG/AS efforts such as DIME and SMART can best be integrated with other related work within the DoD modeling and simulation community to best achieve the vision statement. VISINE activities have focused on participation in the Electronic Defense Simulation Working Group (EDSWG), and transition support to the Joint Modeling and Simulation System (J-MASS) program. While insertion of the DIME software into the Survivability/Vulnerability Information Analysis Center (SURVIAC) later this year will be a definitive step in moving toward a flexible and efficient computational environment, the creation and empowerment of the EDSWG shows real promise in building a "joint Service methodology for conducting air weapon system survivability analysis." [The EDSWG is described in more detail on page 13 of this issue by the chairman, LT Col Kevin Cheek.] VISINE participation in the EDSWG has centered on developing template-based, object-oriented user (analyst) requirements for electronic countermeasures (ECM) effectiveness assessment against threat surface-to-air missile (SAM) systems. Modeling requirements for a class of survivability assessments have thus been codified in a cogent, coherent manner by a truly joint working group, before the new models (J-MASS) are built. This seminal work links a "joint Service methodology" with the computational environment and credible components to support this methodology.
...the creation and empowerment of the EDSWG shows real promise in building a "joint Service methodology for conducting air weapon system survivability analysis."
In a closely-related role, the VISINE project has followed recent efforts to transition J-MASS program control to USAF/XOM and has been a catalyst in inserting a user requirements forum to help focus development of a usable set of J-MASS models. The JTCG/AS Methodology Subgroup has been proposed to chair the J-MASS Users' Group as the source of user (analyst) requirements; how the Methodology Subgroup committee will assume this role is under discussion. At this stage, two points are clear:
- Near-term efforts will use the functional requirements templates developed by the EDSWG to focus J-MASS model development
- Formal functional area developer domains are essential to the proper implementation of the J-MASS concept, in particular to promote and enable software reuse and ensure the consistent and "authoritative representations" of systems called for by DoD Directive 5000.59.
In summary, the JTCG/AS Methodology Subgroup, through the VISINE project, is leading the integration of its efforts with those of the DoD modeling and simulation community to ensure that survivability analysis capabilities match the subgroup vision statement. EDSWG provides the methodology in the form of requirements based on answering specific analytical questions. DIME provides the computational environment in the form of a survivability analyst domain-oriented user interface. SMART delivers near-term credibility to existing modeling components, and J-MASS promises credible components in the form of authoritative representations of the systems involved in survivability assessment. The Methodology Subgroup appears to have all its bases covered.
Sgt. Friday, that old-time favorite TV detective, was known for his deadpan, "Just the facts, ma'am" approach to investigating cases. He may have taken special interest in a complex investigation underway today for air combat methodology. The current FACT that's being investigated is the Future Air Combat Tool. Interest in developing FACT stems from concerns about redundancies or duplications between air combat simulations. The Survivability/Vulnerability Information Analysis Center (SURVIAC) currently is performing a Future Air Combat Tool Requirements study (FACTR) to define user requirements for FACT.
Government agencies and contractors perform air combat modeling for several purposes, ranging from design trade offs to tactics development to weapon systems evaluation. While a variety of tools have evolved to address these requirements, the BRAWLER and AASPEM models are used extensively by a large segment of the analysis community for supporting key system decisions.
BRAWLER was developed primarily by Decision Science Applications, Inc. (DSA), which, until recently, had retained some proprietary rights to the model. The resulting limitation on Government access to the model constrained BRAWLER's distribution. However, the recent removal of proprietary restrictions on the model has enabled it to have wider dissemination; currently there are 30 BRAWLER users. The removal of restrictions also lifted the last barriers to BRAWLER's entry into SURVIAC. BRAWLER is now approved as a SURVIAC model for air combat.
AASPEM has been a SURVIAC model since 1990. Through SURVIAC, AASPEM has enjoyed wide dissemination, but several versions of AASPEM exist. The current version in SURVIAC is AASPEM 3.3M, and AASPEM 4.0 will be released soon. There are 224 AASPEM 3.3M users, 13 AASPEM 4.0 beta site users, and 4 MIL (man-in-the-loop) AASPEM users. Boeing, which developed the MIL AASPEM version, is working on an improved nonproprietary version, which will be called MIL AASPEM II.
BRAWLER and AASPEM are large and complex models. Mastering their use requires dedicated analysts, and updating either model with new tactics or capabilities requires a significant effort. While each model offers some unique features, there is substantial overlap in some simulation functions. The expense associated with maintaining two large models and the apparent increase in overlap of functions drives the desire to develop just one comprehensive model.
The objective of the FACTR study is to provide sufficient data to make an informed decision about transitioning to a single air-to-air combat analysis tool that will best meet the needs of the tri-Service user community. This study has two phases: the first involves developing a comprehensive list of requirements that any single air combat simulation must meet, and the second involves assessing AASPEM and BRAWLER against those requirements. Both assessments include scoping roadmaps for each model to address any shortfalls or deficiencies.
To develop a comprehensive list of air combat simulation requirements, we surveyed current simulation users. We developed a survey questionnaire and reviewed it with the Government points-of-contact (POCs). The survey addressed model requirements for functional, utility, and technical features. The functional features describe the type of problem for which the model is to be used. The utility features describe the hardware and software aspects of the model. The technical features describe what the model will simulate. These areas were chosen as independent aspects of any simulation or model. In some regards, these aspects are similar to the standard model documentation of users, programmers, and analysts manuals.
We sent the survey to all known air combat model users, using SURVIAC as a resource for the model users address list. Surveys were sent to TRAP, BRAWLER, AASPEM, HELIPAC, and BLUEMAX users. We also described the task at several conferences and meetings and handed out survey questionnaires at those meetings.
We tallied the survey results. While the results were not sufficient to define air combat simulation requirements, they did provide the basis for an analysis to produce the requirements. We developed an analysis with prioritized requirements: essential, important, and "nice to have." We reviewed this analysis approach and the resulting ranking of requirements with the Government POCs and model users at respective user meetings. We incorporated comments from these forums into the final air combat simulation requirements report, which will be released soon in draft form for final user comments.
For the second phase of the study, each model was assessed against the approved list of requirements. To gain familiarity with the respective models, SURVIAC attended training sessions and also worked directly with the models. Each model was then assessed as to the degree to which it satisfied the functional, utility, and technical requirements. We identified model deficiencies and will develop roadmaps to address them. The roadmaps will define the tasks to be accomplished and the estimated time and resources required.
We will write a final report assessing BRAWLER and MIL AASPEM II software tools. The report will include the list of requirements and the assessment of each model against the requirements. We also will present a recommended plan with a proposed roadmap to USAF/XOM to support their policy decisions.
Then, just like Sgt. Friday, we will try to stay out of any crossfire.
Today's challenge in systems development is to develop new systems quicker, cheaper, more effectively, and with greater reliability. DoD Instruction 5000.2, Part 6, Section F, "Survivability," states: "As early as practicable, developers and test agencies will assess survivability and validate critical survivability characteristics at as high a system level as possible." In the survivability area, accomplishing this challenge requires the capability to better define survivability objectives and validate associated criteria early in the acquisition process.
Prior to Acquisition Milestone I, during the Concept Exploration and Definition Phase, critical survivability issues are addressed, parameters are established, and values are defined, which can greatly impact aircraft development. These issues, parameters, and values support critical program documentation, which includes the Mission Needs Statement (MNS), System Threat Analysis Report (STAR), Operational Requirements Document (ORD), Acquisition Program Baseline (APB), Test and Evaluation Master Plan (TEMP), Integrated Logistics Support Plan (ILSP), and Cost and Operational Effectiveness Analysis (COEA). Although the documents are organic, or changing, and are updated at each major milestone, system acquisition and system survivability assessments can be accomplished best if the issues, parameters, and values used are based on the most accurate analysis methodology available.
Currently, the methodology available is frequently based on a "best guess" approach, especially given the difficulty of evaluating aircraft survivability in a "full-up" operational environment that includes applicable Red, Blue, and Grey force players. But a state-of-the-art capability, Distributed Interactive Simulation (DIS), can improve this situation.
DIS is a synthetic environment within which people can interact through simulation(s) at multiple sites networked using compliant architecture, modeling, protocols, standards, and databases. DIS enables the system developer and user to simulate the future operational environment. This capability supports development of the required survivability information during the early system development phases. With the appropriate network of compatible man-in-the-loop simulators, the simulated environment could include future potential threats; a representative aircraft system/configuration; planned operational tactics; appropriate Red, Blue, and Grey Force scenario players; and the system operators necessary to address the critical man-in-the-loop influence on survivability.
Using DIS to create a simulation that includes the elements required to initially address the survivability areas associated with an aircraft system would enhance the identification and validation of critical survivability issues, parameters, and values early in the development cycle. Improving the validity of this initial survivability information could significantly reduce the risk of discovering, later in the development cycle, that the early analysis "missed the mark" in the critical survivability areas.
Figure 1. JAST Scenario
A DIS scenario could be used to assess the survivability areas of a future aircraft system, such as the Joint Advanced Strike Technology (JAST) shown in Figure 1. Using a simulation similar to the one depicted would enable access to a virtual environment, helping JAST developers and users identify critical survivability issues and parameters and make the appropriate trade offs in mission area cost effectiveness studies. For example, a simulation with this capability would enable rapid evaluation of scenario-based interactions between the threat and the JAST system to facilitate decisions about the degree of stealth, the balance between stealth and countermeasures, where to invest resources, and how system designs affect aircraft system warfighting performance.
DIS can enable system developers, users, and testers to take a closer look at the system in its operational environment earlier. As DIS development continues and validation and credibility improve, DIS may prove valuable in later system acquisition phases to evaluate aircraft survivability in an integrated threat environment.
Monday, June 12, 1995
5:00-9:00 p.m. Pre-registration: Columbia Inn, Columbia, MD
Tuesday, June 13, 1995
7:00 a.m. Registration & Security Check-in: Kossiakoff Center
SESSION I
INTRODUCTION
Chairman: Mr. Dale B. Atkinson
8:00 a.m. Welcome and Opening Remarks: Mr. Dale B. Atkinson, Symposium Chairman
8:15 a.m. Keynote Address: Speaker TBA
8:45 a.m. Air Combat Survivability - The Army View: MG John S. Cowings, USA, Commander, U.S. Army Aviation and Troop Command and Chairman, Joint Aeronautical Commanders Group (JACG)
9:15 a.m. Air Combat Survivability - The Navy View: VADM William C. Bowes, USN, Principal Deputy Assistant Secretary of the Navy (Research, Development, and Acquisition) and Former JACG Navy Representative
9:45 a.m. BREAK
10:05 a.m. Air Combat Survivability - The Air Force View: LT Gen Richard M. Scofield, USAF, Commander, Aeronautical Systems Center and JACG Air Force Representative
10:35 a.m. History of Air Combat Survivability: Dr. Robert E. Ball, Distinguished Professor, Naval Postgraduate School and Mr. Dale B. Atkinson, Symposium Chairman
11:05 a.m. Air Combat Survivability - State of the Discipline: Mr. David P. Hornick, Director, Survivability Division, Naval Air Systems Command, and Chairman, JTCG/AS
11:30 a.m. LUNCH - Luncheon Speaker: Mr. Skip Ringo, Manager, Aircraft Engines and Legislative Affairs, General Electric Company
1:00 p.m. Future Threat Briefing: Mr. David Turich, Global Threat Directorate, National Air Intelligence Center
SESSION II
AVOIDING THE HIT - SUSCEPTIBILITY REDUCTION
Chairman: COL Roy P. Oler, Program Manager, Aviation Electronic Combat; Army Principal Member, JTCG/AS
1:30 p.m. Opening Remarks: Session Chairman
1:40 p.m. Susceptibility Reduction Overview: Mr. Lawrence P. De Cosimo, Chief, EW Systems Development Team, Night Vision and Electro-Optic Systems Directorate, U.S. Army CECOM; Chairman, Susceptibility Reduction Subgroup, JTCG/AS
1:55 p.m. Low Observables: Mr. William Bahret, Private Consultant
2:40 p.m. Smart Munitions: LTC William D. Horton, USA, Assistant Program Manager for Electronic Warfare, Program Manager for Survivability Systems
3:10 p.m. BREAK
3:30 p.m. Foreign Missile Technology: Mr. Don Fetterly, Project Engineer, Foreign Technology, Hughes Missile Systems Company
4:00 p.m. Imaging Seeker Countermeasures: Dr. Frank Barone, Section Head, IRCM Techniques Section, Optical Science Division, Naval Research Laboratory; Co-Chairman, Susceptibility Reduction Subgroup
4:30 p.m. OSD Technology Perspective: Mr. Stanley E. Gontarek, Staff Specialist for Electronic Warfare and C3I Technology Office of the Director, Defense Research and Engineering (Advanced Technology), OUSD (A&T)
5:00 p.m. Adjourn
5:00-7:30 p.m. Reception/Exhibits - Kossiakoff Center
Wednesday, June 14, 1995
SESSION III
SURVIVING THE HIT - VULNERABILITY REDUCTION
Chairman: Mr. Richard R. Ledesma, Deputy Director, Test, Systems Engineering and Evaluation (Air and Space Programs), OUSD (A&T)
8:00 a.m. Opening Remarks: Session Chairman
8:15 a.m. Vulnerability Reduction - State-of-the-Art: Mr. Richard A. Horton, Head, Survivability Division, Naval Air Warfare Center Weapons Division, China Lake; Chairman, Vulnerability Reduction Subgroup, JTCG/AS
8:45 a.m. Fuel System/Dry Bay Alternative Concepts and Testing: Mr. J. Hardy Tyson, Team Leader, F/A-18 E/F Vulnerability, Naval Air Warfare Center Weapons Division, China Lake
9:20 a.m. Fuel System/Dry Bay Protection: Mr. J. Michael Bennett, Chief, Technology Group, Survivability and Safety Branch, U.S. Air Force Wright Laboratory
9:50 a.m. BREAK
10:10 a.m. Integrating Vulnerability Reduction into Combat Aircraft Design: Mr. Levelle Mahood, Senior Technical Specialist, Survivability, Advanced Design, Northrop Grumman
10:40 a.m. Aircraft Vulnerability Reduction - Commercial Aircraft Payoff: Mr. Rick Peters, Program Manager for the Transport Aircraft Survivability Program, U.S. Air Force Wright Laboratory and Mr. Joseph Gatto, Aerospace Engineer, FAA Aircraft Hardening Program
11:15 a.m. Vulnerability Reduction - The Future: Mr. Anthony S. Lizza, Section Chief, Fatigue, Fracture, and Reliability Branch, U.S. Air Force Wright Laboratory; Co-Chairman, Vulnerability Reduction Subgroup, JTCG/AS
11:30 a.m. LUNCH
SESSION IV
ASSESSING THE HIT - SURVIVABILITY METHODOLOGY
Chairman: Mr. D. Jerry Wallick, Logistics Management Institute, Inc.; Chairman, Survivability Technical Committee, AIAA
1:00 p.m. Opening Remarks: Session Chairman
1:15 p.m. Methodology Overview: Mr. Martin L. Lentz, Flight Dynamics Directorate, U.S. Air Force Wright Laboratory; Co-Chairman, Survivability Methodology Subgroup
1:45 p.m. Susceptibility Model Assessment and Range Test (SMART): Dr. Paul R. Muessig, Deputy Program Manager, SMART Program Office
2:15 p.m. RADGUNS - A JTCG/AS Success Story: Mr. Dwight G. Fitzsimons, Program Manager, RADGUNS, National Ground Intelligence Center
2:35 p.m. Future Air Combat Tool Requirements (FACTR): Mr. David R. Mittlesteadt, USAF Aeronautical Systems Center, and LT Col Marty Allen, USAF, Air Force Studies and Analysis Agency
2:55 p.m. BREAK
3:15 p.m. AJEM - The TILV and JTCG/ME Connection: Mr. Thomas L. Wasmund, Naval Surface Warfare Center - Dahlgren
3:35 p.m. DIME, J-MASS and EDSWG - How Does It All Fit Together?: Mr. Robert J. Meyer, Naval Air Warfare Center Weapons Division, China Lake and LT Col Kevin Cheek, USAF, Chief, Force Employment Branch, Force Application Division, Air Force Studies and Analysis Agency
4:05 p.m. Where Do We Go From Here?: Mr. David H. Hall, Chief Analyst, Survivability Division, Naval Air Warfare Center, China Lake; Chairman, Survivability Methodology Subgroup, JTCG/AS
4:35 p.m. Adjourn
Thursday, June 15, 1995
SESSION V
JOINT LIVE FIRE (JLF) and BATTLE DAMAGE REPAIR (BDR)
Chairman: Mr. James F. O'Bryon, Office of the Director, OT&E, OSD
8:00 a.m. Opening Remarks: Session Chairman
8:15 a.m. Joint Live Fire - Now and in the Future: Mr. Ralph W. Lauzze II, Joint Test Director, Joint Live Fire (Aircraft) Program; Air Force Principal Member, JTCG/AS
8:40 a.m. Joint Live Fire - Aircraft Weapons Bay Vulnerability Test and Analysis: Mr. Alex G. Kurtz, Survivability and Safety Branch, U.S. Air Force Wright Laboratory, and Mr. Leo Budd, Survivability Division, Naval Air Warfare Center Weapons Division, China Lake
9:10 a.m. B-2 Battle Damage Repair Program: Mr. Brian Kilburn, ABDR Engineer, B-2 Systems Support Team, Oklahoma Air Logistics Center, Tinker AFB, OK
9:35 a.m. V-22 Battle Damage Repair Program Plan: Mr. Rob Hartel, V-22 Airframes Engineer, Product Support Division, Cherry Point, NC
10:00 a.m. BREAK
SESSION VI
THE NEXT 25 YEARS
Chairman: RADM Robert H. Gormley, USN (Ret.), Chairman, Combat Survivability Division, ADPA
10:20 a.m. Panel Discussion: Challenges in Air Combat Survivability - The Next 25 Years
Moderator: RADM Robert H. Gormley, USN (Ret.), Chairman, Combat Survivability Division, ADPA
Panel Members:
Mr. Lee H. Frame, Deputy Director, Operational Test and Evaluation (Conventional Systems), OSD
Brig Gen William S. Hinton, Jr., USAF, Director of Requirements, Air Combat Command
Mr. Frank Kendall, Vice President-Engineering, Raytheon Company; Former Director, Tactical Warfare Programs, OUSD (A&T)
Mr. Bartley P. Osborne, Jr., Vice President-Engineering, Lockheed Aeronautical Systems Company
Mr. John Porter, President, Association of Old Crows; Former Director, Electronic Warfare, OSD
11:45 p.m. LUNCH
SESSION VII
THE PAYOFF - SYSTEMS SURVIVABILITY APPLICATIONS
Chairman: Dr. Albert E. Rainis, Staff Specialist for Survivability, OUSD (A&T)
12:45 p.m. Opening Remarks: Session Chairman
1:00 p.m. V-22 Survivability Program: Col Robert Garner, USMC, V-22 Program Manager
1:30 p.m. F-22 Survivability Program: Mr. Jon Ogg, F-22 Chief Engineer, F-22 System Program Office
2:00 p.m. RAH-66 Survivability Program: COL Randy Oliver, USA, Commander, Aviation Applied Technology Directorate, U.S. Army Aviation and Troop Command
2:30 p.m. BREAK
2:50 p.m. F/A-18 E/F Survivability Program: CAPT Joe Dyer, USN, F/A-18 Program Manager
3:20 p.m. C-17 Survivability Program: Mr. Rick Abel, Technical Director, C-17 System Program Office
3:50 p.m. JAST - Survivability for the Future: Dr. Don McErlean, Technology Maturation Director, JAST Program
4:20 p.m. Closing Remarks: Mr. Dale B. Atkinson, Symposium Chairman
THE JOINT AERONAUTICAL COMMANDERS GROUP
JOINT TECHNICAL COORDINATING GROUP ON AIRCRAFT SURVIVABILITY
NAVAL AIR SYSTEMS COMMAND
1421 JEFFERSON DAVIS HIGHWAY
ARLINGTON, VA 22243-5120
On behalf of the Joint Aeronautical Commanders Group (JACG), we invite you to join us for the 1995 Air Combat Survivability Symposium. This gathering of aircraft survivability specialists will be held June 13-15 at the Johns Hopkins University's Kossiakoff Center in Laurel, Maryland.
The theme of our symposium, the first of its kind in over 7 years, is "Challenges in Air Combat Survivability: The Next 25 Years." Our specific objectives include identifying the challenges faced by the aircraft survivability discipline over the next quarter century, examining the technological state of the art, and highlighting current and future research efforts. This event is being sponsored by the Joint Technical Coordinating Group on Aircraft Survivability (JTCG/AS), with participation by the Survivability Technical Committee of the American Institute of Aeronautics and Astronautics (AIAA) and the Combat Survivability Division of the American Defense Preparedness Association (ADPA).
The JTCG/AS was originally formed in 1971 in response to large aircraft losses in Southeast Asia. It serves as the tri-Service focal point for aircraft combat survivability issues. The JTCG/AS is chartered by the Joint Aeronautical Commanders Group, with funding and technical oversight from the Office of the Undersecretary of Defense (Acquisition and Technology). The organization coordinates and conducts joint aircraft survivability research and development, and it has played a major role in establishing survivability as a design discipline. The efforts of the JTCG/AS have been credited with contributing to the light aircraft losses experienced during Operation Desert Storm.
The discipline of aircraft survivability deals with the capability of an aircraft to avoid and/or withstand a man-made hostile environment and is considered in terms of susceptibility reduction and vulnerability reduction. Susceptibility reduction includes those efforts that decrease the likelihood of an aircraft being detected, acquired, tracked, engaged, and hit by a hostile threat, expressed as the probability of a hit. Vulnerability reduction deals with minimizing the likelihood of an aircraft being rendered incapable of completing its combat mission as a result of such a hit and is described in terms of the probability of a kill, given a hit.
Our agenda features a distinguished group of speakers representing a broad cross section of the air combat survivability community, and we expect to attract several hundred survivability professionals from both government and industry. Seats are limited, so we encourage you to register early. To maximize the exchange of current and meaningful information, this conference will be classified SECRET/NOFORN.
We look forward to seeing you at the 1995 Air Combat Survivability Symposium. Your participation will help to make this a truly world-class professional session.
Sincerely,
Dale B. Atkinson
Symposium Chairman
LTC John N. Lawless, Jr., USA
Symposium Coordinator
Information for inclusion in the Calendar of Events may be sent to:
Event: SURVIAC Survivability Analysis Workshop
Date: 17-21 Apr 95
Location: Wright-Patterson AFB, OH
POC: Sue Green, (513) 255-4840
Event: RADGUNS Users Meeting
Date: 25-27 Apr 95
Location: Charlottesville, VA
POC: Susan Olson, (804) 980-7716
Event: JTCG/AS Air Combat Survivability Symposium - "Challenges in Air Combat Survivability: The Next 25 Years" (SECRET/NOFORN)
Date: 13-15 Jun 95
Location: Laurel, MD
POC: Diane Lussier, (703) 902-4882
Event: 30th AIAA/SAE/ASME/ASEE Joint Propulsion Conference & Exhibit
Date: 10-12 Jul 95
Location: San Diego, CA
POC: Jennifer Reins, (202) 646-7463
Event: AUVS '95 - Unmanned Systems Around the World
Date: 10-12 Jul 95
Location: Washington, DC
POC: Christine Murphy, (703) 524-6646
Event: AIAA Guidance, Navigation, and Control Conference
Date: 7-10 Aug 95
Location: Baltimore, MD
POC: Jennifer Reins, (202) 646-7463
Event: Aircraft Fire Protection/Mishap Investigation Course
Date: 7-11 Aug 95
Location: Dayton, OH
POC: Robert Clodfelter, (513) 435-8778
Event: 1st AIAA Aircraft Engineering, Technology, and Operations Congress
Date: 19-21 Sep 95
Location: Los Angeles, CA
POC: Jennifer Reins, (202) 646-7463
