Thursday, December 20, 2012

FDS 6 Verification and Validation

In recent years, we have formalized the process of developing and releasing new versions of FDS and Smokeview, taking our cue from the way it is done by commercial software developers. The key to this process is a set of calculations that we divide into two categories -- relatively short, simple test cases that we use to verify that the programs are properly solving the equations we've written down, and relatively long, sometimes complex cases that we use to validate the accuracy of the model by comparing with actual experiments.

The short verification cases are the sample input files that we distribute with each new bundle. They are run each night automatically by a script we call 'firebot' to catch simple coding mistakes that we inevitably make as we develop new routines. We not only run these cases, but we also automatically regenerate all the plots that you find in the FDS and Smokeview User's Guide, plus the FDS Verification Guide. Any result that falls outside of a certain tolerance is reported to all of us via email, and we then fix the problem while it is still fresh on our minds.

The longer validation cases are run with each minor release of FDS and Smokeview because it takes between one and two weeks to complete all the cases (about 800 calculations) on our 256 core linux cluster. Because these cases are run less frequently, there almost always is something amiss about one or two cases that is more difficult to diagnose. In fact, a good part of the two years we spent preparing FDS 6 was devoted to addressing discrepancies in some of our most reliable experimental data sets. As frustrating as it was, the fact that we knew about it all was due to the new process of V&V that we initiated. In the past, it was far more difficult to detect a fundamental problem with the algorithm because we did not systematically run all the cases at once and analyze the results in a consistent way. Basically, we just eye-balled things, but that does not always reveal subtle problems.

When we released FDS version 1 in 2000, we did not have a formal process of V&V. We did develop test cases, and we did compare calculations with experiment, but we did not do it in a systematic way. We felt that papers published by ourselves and others would suffice, but we soon learned that this was not the case. This lesson was reinforced when we began working with the US Nuclear Regulatory Commission on a V&V of five different fire models that are commonly used by the nuclear industry. The most important lesson we learned is that published results using older versions of the software cannot be used to justify the use of a model to the AHJ (Authority Having Jurisdiction). We cannot republish our validation papers with the release of each new version, so we decided to develop and maintain our own versions of the V&V guides that the NRC published (NUREG-1824).

Now the FDS Verification and Validation Guides are the key volumes that quantify the robustness and accuracy of the model. When we say that FDS can or cannot do something, what we really mean is that we have documented calculations that show the range and accuracy of the model for a particular application. For those of you using FDS for design or forensic applications, the FDS V&V Guides should be the first place to look to determine if FDS is appropriate for your use. Every few days someone writes to the Discussion Group asking something like, "Does anybody know if FDS can do ...?" What that person really ought to do is check the V&V Guides. The question should not only be can it be done, but also how well can it be done. FDS has alot of potential applications, but you need to check the Guides before deciding if it is applicable.

Finally, we spend a considerable amount of work putting the V&V Guides together, running the cases, working on the statistics, the scripts, and so on. We have appealed to the user community, especially the students and profs working with FDS, to help us with V&V cases. So far, the results have been disappointing. For a variety of reasons, we are not able to capture in our Guides the work with FDS that we see submitted to journals and conferences. We suspect that the main reason for this is that a student's first objective is to graduate, then maybe publish the thesis in some way. Working with us to get the results into our Guides is either a low priority or something that no one has even considered. I sometimes meet students at meetings who do not think that their thesis work is worthy of our Guides. Think again -- not only would the work be worthy in most cases, but it is a necessity. In order for us all to continue to develop and use tools like FDS and Smokeview, we all have to contribute to their upkeep. In this case, that means extending the range of application represented by the V&V cases. If you did your thesis on smoke detector algorithms, it is in your best interest to get this work into the Guides so that we can maintain the capability in future versions. If you just hack some routine into FDS 5.3.whatever, don't expect it to be accepted down the road by the AHJ. Further, owing to the relatively small size of fire protection engineering, there are probably few other engineering disciplines where a masters degree student can have his or her work put directly into practice so readily. Ask engineers in other disciplines what kinds of software packages they use and whether or not they have any means to be involved in their development. One of the appealing things about FPE is that there are so many opportunities to have an impact. To see how you can have an impact, read through the following wiki:

Friday, December 14, 2012

FDS 6 RC2 and Defining Species in FDS 6

To all of you who have provided feedback on FDS 6 RC1, thank you.  Based on feedback we have made some changes to inputs, fixed a few bugs, and have now released FDS 6 RC2.  You can download FDS 6 RC2 from the list of all available downloads.   A major set of changes from RC1 to RC2 is the specification of species.  Based on feedback on the inputs for the new lumped species capability, we have removed the SMIX input and moved the definition of lumped species to the SPEC input.

The remainder of this post will be another in our continuing series on topics in the FDS 6 Release Notes.  This post will focus on some significant changes made to species definitions in FDS 6 including the recent changes in RC2.

FDS 2 through the early versions of FDS 5 gave two options to tracking species.  You could define a primitive species that was tracked by FDS (like the water vapor from sprinklers), or you could make use of a reaction progress variable (which we termed at the time the mixture fraction).  In FDS 5.2 we began shifting, under the hood, the tracking of species from the conserved progress variables to a lumped species approach.  A lumped species is simply an equivalent gas species that represents a mixture of gasses that are always transported together in the same ratio.  For example, air can be considered as a lumped species consisting of 79 % N2 21 % O2 (and trace amounts of other gasses).  As input, however, you could still only use primitive species or access predefined lumped species by using a simple REAC line input.  In this case the predefined species were air and products.  The species were defined based upon the fuel you specified and the single-step reaction of:

Fuel + Air -> Products

For example if the fuel was propane you had:

C3H8 + 5 (O2 + 3.76 N2) ->3 CO2 + 4 H2O + 5 (3.76) N2

In this case the Air species would be the N2+O2 mixture on the left hand side of the equation (the actual species also has ambient CO2 and H2O) and the Product species was the mixture of gasses on the right hand side of the equation.

Doing this was motivated by a few factors.  First, by taking this approach we no longer needed to make use of complex state relationships to obtain the mass fractions of gasses.  Instead, simply multiplying the vector of lumped species by a transformation matrix would give the primitive species. Second, there were requests to track toxicant and irritant products (HCl, HCN, etc.) in order to compute FED and FIC. Trying to provide flexibility to add additional species within the progress variable framework would have been more difficult than the lumped species approach. Lastly, there were other development efforts to make the combustion model more flexible (which will be the subject of a future blog post) and those would be aided by a more flexible set of species inputs. 

What does all this mean for you?  If all you wish to do is use the REAC inputs for simple chemistry with the single-step combustion that was in FDS 5, the changes to species will remain under the hood.  Moving beyond the predefined species present in FDS 5, you now have the ability to define your own.  You now have two types of species that you can input, a primitive species (a single gas) or a lumped species.  For example adding the line shown below will cause FDS to track the species H2.


FDS 6 comes with a much larger set of predefined primitive species, but FDS 6 also allows you to define your own species with better control over the properties than in FDS 5.  You can also define the chemistry of the species which will enable FDS to automatically compute is molecular weight and the formula can then be used with the new REAC capabilities. For exampe:


To input a lumped species, you must first define all the primitive species and then define how those species combine to form the lumped species.  For example, the lines below create a lumped species called AIR that consists of 79 % N2 and 21 % O2.  The LUMPED_COMPONENT_ONLY=.TRUE. tells FDS that the primitive species of NITROGEN and OXYGEN are only found in a lumped species and that FDS should not allocate memory to track them separately.:


Consider this set of inputs:

The above defines three lumped species that are all H2S.  Consider a case where you needed to simulate small leaks of H2S from a variety of sources.  If the leaks are small enough that they don't change the normal ambient airflow, then you could simulate multiple leaks at once provided you had a method to distinguish one leak from another.  With the lines above you can easily define a set of species that have the same properties, but are tracked separately.  If you were to use SPEC_ID='H2S-1' on an output, you would just get that species.  If you were to use SPEC_ID='HYDROGEN SULFIDE' you would get the sum of all three of the lumped species.

In addition to the changes discussed above there are three other notable improvements to the &SPEC input.  First, you can now define temperature dependent properties for custom species (FDS 5 limited you to constant properties). Second, the definition of liquid properties has been moved from &PART to &SPEC so that all species data is processed in one location.  Third, we have added the ability to define some simple aerosol properties.  This last was done to support recent efforts in adding aerosol deposition models to FDS (to compute soot deposition).