Software:American fuzzy lop (fuzzer)

From HandWiki
Short description: Software fuzzer that employs genetic algorithms
American fuzzy lop
American fuzzy lop's afl-fuzz running on a test program.png
American fuzzy lop's afl-fuzz running on a test program
Developer(s)Michał Zalewski
Initial release12 November 2013; 10 years ago (2013-11-12)
Stable release
2.57b / 30 June 2020; 4 years ago (2020-06-30)[1]
Written inC, assembly
Operating systemCross-platform
TypeFuzzer
LicenseApache License 2.0

American fuzzy lop (AFL), stylized in lowercase as american fuzzy lop, is a free software fuzzer that employs genetic algorithms in order to efficiently increase code coverage of the test cases. So far it has detected dozens of significant software bugs in major free software projects, including X.Org Server,[2] PHP,[3] OpenSSL,[4][5] pngcrush, bash,[6] Firefox,[7] BIND,[8][9] Qt,[10] and SQLite.[11]

For many years after its release, AFL has been considered a "state of the art" fuzzer.[12] AFL is considered "a de-facto standard for fuzzing",[13] and the release of AFL contributed significantly to the development of fuzzing as a research area.[14] AFL is widely used in academia; academic fuzzers are often forks of AFL,[15] and AFL is commonly used as a baseline to evaluate new techniques.[16][17]

The source code of American fuzzy lop is published on GitHub. Its name is a reference to a breed of rabbit, the American Fuzzy Lop.

Overview

AFL requires the user to provide a sample command that runs the tested application and at least one small example input. The input can be fed to the tested program either via standard input or as an input file specified in the process command line. Fuzzing networked programs is currently not directly supported, although in some cases there are feasible solutions to this problem.[18] For example, in case of an audio player, American fuzzy lop can be instructed to open a short sound file with it. Then, the fuzzer attempts to actually execute the specified command and if that succeeds, it tries to reduce the input file to the smallest one that triggers the same behavior.

After this initial phase, AFL begins the actual process of fuzzing by applying various modifications to the input file. When the tested program crashes or hangs, this usually implies the discovery of a new bug, possibly a security vulnerability. In this case, the modified input file is saved for further user inspection.

In order to maximize the fuzzing performance, American fuzzy lop expects the tested program to be compiled with the aid of a utility program that instruments the code with helper functions which track control flow. This allows the fuzzer to detect when the target's behavior changes in response to the input. In cases when this is not possible, black-box testing is supported as well.

Fuzzing algorithm

AFL's logo from fuzzed input stitched together as a single animation.[19]

Fuzzers attempt to find unexpected behaviors (i.e., bugs) in a target program by repeatedly executing the program on various inputs. As described above, AFL is a gray-box fuzzer, meaning it injects instrumentation to measure code coverage into the target program at compile time and uses the coverage metric to direct the generation of new inputs. AFL's fuzzing algorithm has influenced many subsequent gray-box fuzzers.[20][21]

The inputs to AFL are an instrumented target program (the system under test) and corpus, that is, a collection of inputs to the target. Inputs are also known as test cases. The algorithm maintains a queue of inputs, which is initialized to the input corpus. The overall algorithm works as follows:[22]

  1. Load the next input from the queue
  2. Minimize the test case
  3. Mutate the test case. If any mutant results in additional code coverage, add it to the queue. If the mutant results in a crash or hang, save it to disk for later inspection.
  4. Go to step 1

Mutation

To generate new inputs, AFL applies various mutations to existing inputs.[23] These mutations are mostly agnostic to the input format of the target program; they generally treat the input as simple blob of binary data.

At first, AFL applies a deterministic sequence of mutations to each input. These are applied at various offsets in the input. They include:[24][25]

  • Flipping (i.e., negating or inverting) 1-32 bits
  • Incrementing and decrementing 8-, 16-, and 32-bit integers, in both little- and big-endian encodings
  • Overwriting parts of the input with "approximately two dozen 'interesting' values", including zero and maximum and minimum signed and unsigned integers of various widths, again in both little- and big-endian encodings.
  • Replacing parts of the input with data drawn from a "dictionary" of user-specified or auto-detected tokens (e.g., magic bytes, or keywords in a text-based format)[26][23][27]

After applying all available deterministic mutations, AFL moves on to havoc, a stage where between 2 and 128 mutations are applied in a row. These mutations are any of:[23]

  • The deterministic mutations described above
  • Overwriting bytes with random values
  • Operations over multi-byte "blocks":
    • Deleting blocks
    • Duplicating blocks
    • Setting each byte in a block to a single value

If AFL cycles through the entire queue without generating any input that achieves new code coverage, it begins splicing. Splicing takes two inputs from the queue, truncates them at arbitrary positions, concatenates them together, and applies the havoc stage to the result.

Measuring coverage

AFL pioneered the use of binned hitcounts for measuring code coverage.[28] The author claims that this technique mitigates path explosion.[29][30]

Conceptually, AFL counts the number of times a given execution of the target traverses each edge in the target's control-flow graph; the documentation refers to these edges as tuples and the counts as hitcounts. At the end of the execution, the hitcounts are binned or bucketed into the following eight buckets: 1, 2, 3, 4-7, 8-15, 16-31, 32-127, and 128 and greater. AFL maintains a global set of (tuple, binned count) pairs that have been produced by any execution thus far. An input is considered "interesting" and is added to the queue if it produces a (tuple, binned count) pair that is not yet in the global set.

In practice, the hitcounts are collected and processed using an efficient but lossy scheme. The compile-time instrumentation injects code that is conceptually similar to the following at each branch in the control-flow graph of the target program:[31] 

cur_location = <COMPILE_TIME_RANDOM>;
shared_mem[cur_location ^ prev_location]++;
prev_location = cur_location >> 1;

where <COMPILE_TIME_RANDOM> is a random integer and shared_mem is a 64 kilobyte region of memory shared between the fuzzer and the target.

This representation is more fine-grained (distinguishes between more executions) than simple block or statement coverage, but still allows for a linear-time "interestingness" test.

Minimization

On the assumption that smaller inputs take less time to execute, AFL attempts to minimize or trim the test cases in the queue.[23][32] Trimming works by removing blocks from the input; if the trimmed input still results in the same coverage (see #Measuring coverage), then the original input is discarded and the trimmed input is saved in the queue.

Scheduling

AFL selects a subset of favored inputs from the queue, non-favored inputs are skipped with some probability.[33][28]

Features

Performance features

One of the challenges American fuzzy lop had to solve involved an efficient spawning of hundreds of processes per second. Apart from the original engine that spawned every process from scratch, American fuzzy lop offers the default engine that relies heavily on the fork system call.[34][28] This can further be sped up by leveraging LLVM deferred fork server mode or the similar persistent mode, but this comes at the cost of having to modify the tested program.[35] Also, American fuzzy lop supports fuzzing the same program over the network.

User interface

American fuzzy lop features a colorful command line interface that displays real-time statistics about the fuzzing process. Various settings may be triggered by either command line options or environment variables. Apart from that, programs may read runtime statistics from files in a machine-readable format.

Utility programs

In addition to afl-fuzz and tools that can be used for binary instrumentation, American fuzzy lop features utility programs meant for monitoring of the fuzzing process. Apart from that, there is afl-cmin and afl-tmin, which can be used for test case and test corpus minimization. This can be useful when the test cases generated by afl-fuzz would be used by other fuzzers.

Forks

AFL has been forked many times in order to examine new fuzzing techniques, or to apply fuzzing to different kinds of programs. A few notable forks include:

  • AFL++
  • MOPT-AFL[36]
  • AFLFast[37]
  • AFLSmart[38]
  • AFLGo[39]
  • SymCC-AFL[40]
  • WinAFL, "a fork of AFL for fuzzing Windows binaries"[41]

AFL++

AFL++
Initial release2.52c / 5 June 2019; 5 years ago (2019-06-05)
Stable release
4.00c / 26 January 2022; 2 years ago (2022-01-26)[42]

AFL++ (AFLplusplus)[43] is a community-maintained fork of AFL created due to the relative inactivity of Google's upstream AFL development since September 2017. It includes new features and speedups.[44]

Google's OSS-Fuzz initiative, which provides free fuzzing services to open source software, replaced its AFL option with AFL++ in January 2021.[45][46]

References

Notes

  1. ↑ "Releases - google/AFL". https://github.com/google/AFL/releases. 
  2. ↑ "Advisory-2015-03-17". x.org. http://www.x.org/wiki/Development/Security/Advisory-2015-03-17/. 
  3. ↑ "NVD - Detail". nist.gov. https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-9427. 
  4. ↑ "NVD - Detail". nist.gov. https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2015-0208. 
  5. ↑ "NVD - Detail". nist.gov. https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2015-0288. 
  6. ↑ "CVE - CVE-2014-6278". mitre.org. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-6278. 
  7. ↑ "CVE - CVE-2014-8637". mitre.org. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-8637. 
  8. ↑ "How to fuzz a server with American Fuzzy Lop". Fastly. https://www.fastly.com/blog/how-to-fuzz-server-american-fuzzy-lop. 
  9. ↑ "CVE - CVE-2015-5477". mitre.org. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-5477. 
  10. ↑ "[Announce Qt Project Security Advisory - Multiple Vulnerabilities in Qt Image Format Handling"]. qt-project.org. http://lists.qt-project.org/pipermail/announce/2015-April/000067.html. 
  11. ↑ "How SQLite Is Tested # 4.1.1. SQL Fuzz Using The American Fuzzy Lop Fuzzer". sqlite.org. https://www.sqlite.org/testing.html#sql_fuzz_using_the_american_fuzzy_lop_fuzzer. 
  12. ↑ Poncelet, Clement; Sagonas, Konstantinos; Tsiftes, Nicolas (2023-01-05). "So Many Fuzzers, So Little Time✱: Experience from Evaluating Fuzzers on the Contiki-NG Network (Hay)Stack". Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering. ASE '22 (New York, NY, USA: Association for Computing Machinery): 1–12. doi:10.1145/3551349.3556946. ISBN 978-1-4503-9475-8. https://doi.org/10.1145/3551349.3556946. 
  13. ↑ Fioraldi et al. 2023, p. 2.
  14. ↑ Fioraldi, Andrea; Maier, Dominik Christian; Zhang, Dongjia; Balzarotti, Davide (2022-11-07). "LibAFL: A Framework to Build Modular and Reusable Fuzzers". Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security. CCS '22 (New York, NY, USA: Association for Computing Machinery): 1051–1065. doi:10.1145/3548606.3560602. ISBN 978-1-4503-9450-5. https://doi.org/10.1145/3548606.3560602. . "The release of AFL marked an important milestone in the area of software security testing, revitalizing fuzzing as a major research topic".
  15. ↑ Fioraldi, Andrea; Maier, Dominik Christian; Zhang, Dongjia; Balzarotti, Davide (2022-11-07). "LibAFL: A Framework to Build Modular and Reusable Fuzzers". Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security. CCS '22 (New York, NY, USA: Association for Computing Machinery): 1051–1065. doi:10.1145/3548606.3560602. ISBN 978-1-4503-9450-5. https://doi.org/10.1145/3548606.3560602. . "Many of these studies implemented their techniques by forking the AFL codebase."
  16. ↑ Hazimeh, Ahmad; Herrera, Adrian; Payer, Mathias (2021-06-15). "Magma: A Ground-Truth Fuzzing Benchmark". Proceedings of the ACM on Measurement and Analysis of Computing Systems 4 (3): 49:1–49:29. doi:10.1145/3428334. https://doi.org/10.1145/3428334. 
  17. ↑ Metzman et al. 2021.
  18. ↑ Technion. "Fuzzing nginx - Hunting vulnerabilities with afl-fuzz". lolware.net. https://lolware.net/2015/04/28/nginx-fuzzing.html. 
  19. ↑ Zalewski, Michał (2015-02-27). "Logo for afl-fuzz". https://groups.google.com/d/msg/afl-users/_kNrt_n5Nug/vcZvl2Y7C7EJ. 
  20. ↑ Fioraldi et al. 2023.
  21. ↑ Chen, Peng; Chen, Hao (May 2018). "Angora: Efficient Fuzzing by Principled Search". 2018 IEEE Symposium on Security and Privacy (SP): 711–725. doi:10.1109/SP.2018.00046. ISBN 978-1-5386-4353-2. https://ieeexplore.ieee.org/document/8418633. 
  22. ↑ "Motivation behind AFL — AFL 2.53b documentation". https://afl-1.readthedocs.io/en/latest/motivation.html#the-afl-approach. 
  23. ↑ 23.0 23.1 23.2 23.3 Fioraldi et al. 2023, p. 6.
  24. ↑ "Binary fuzzing strategies: what works, what doesn't". lcamtuf.blogspot.com. http://lcamtuf.blogspot.com/2014/08/binary-fuzzing-strategies-what-works.html. 
  25. ↑ "AFL User Guide — AFL 2.53b documentation". https://afl-1.readthedocs.io/en/latest/user_guide.html#stage-progress. 
  26. ↑ "Finding bugs in SQLite, the easy way". lcamtuf.blogspot.com. http://lcamtuf.blogspot.com/2015/04/finding-bugs-in-sqlite-easy-way.html. 
  27. ↑ Manès, Valentin J.M.; Han, HyungSeok; Han, Choongwoo; Cha, Sang Kil; Egele, Manuel; Schwartz, Edward J.; Woo, Maverick (November 2021). "The Art, Science, and Engineering of Fuzzing: A Survey". IEEE Transactions on Software Engineering 47 (11): 2312–2331. doi:10.1109/TSE.2019.2946563. ISSN 1939-3520. https://ieeexplore.ieee.org/document/8863940. 
  28. ↑ 28.0 28.1 28.2 Fioraldi et al. 2023, p. 5.
  29. ↑ "Technical "whitepaper" for afl-fuzz". https://lcamtuf.coredump.cx/afl/technical_details.txt. 
  30. ↑ "More about AFL — AFL 2.53b documentation". https://afl-1.readthedocs.io/en/latest/about_afl.html?highlight=favored#detecting-new-behaviors.  "This approach allows for a very fine-grained and long-term exploration of program state while not having to perform any computationally intensive and fragile global comparisons of complex execution traces, and while avoiding the scourge of path explosion."
  31. ↑ "More about AFL — AFL 2.53b documentation". https://afl-1.readthedocs.io/en/latest/about_afl.html#coverage-measurements. 
  32. ↑ "More about AFL — AFL 2.53b documentation". https://afl-1.readthedocs.io/en/latest/about_afl.html#trimming-input-files. 
  33. ↑ "More about AFL — AFL 2.53b documentation". https://afl-1.readthedocs.io/en/latest/about_afl.html?highlight=favored#culling-the-corpus. 
  34. ↑ "Fuzzing random programs without execve()". lcamtuf.blogspot.com. http://lcamtuf.blogspot.com/2014/10/fuzzing-binaries-without-execve.html. 
  35. ↑ "New in AFL: persistent mode". 11 June 2015. https://lcamtuf.blogspot.com/2015/06/new-in-afl-persistent-mode.html. 
  36. ↑ Lyu, Chenyang; Ji, Shouling; Zhang, Chao; Li, Yuwei; Lee, Wei-Han; Song, Yu; Beyah, Raheem (2019) (in en). {MOPT}: Optimized Mutation Scheduling for Fuzzers. pp. 1949–1966. ISBN 978-1-939133-06-9. https://www.usenix.org/conference/usenixsecurity19/presentation/lyu. 
  37. ↑ Böhme, Marcel; Pham, Van-Thuan; Roychoudhury, Abhik (May 2019). "Coverage-Based Greybox Fuzzing as Markov Chain". IEEE Transactions on Software Engineering 45 (5): 489–506. doi:10.1109/TSE.2017.2785841. ISSN 1939-3520. https://ieeexplore.ieee.org/document/8233151. 
  38. ↑ Pham, Van-Thuan; Böhme, Marcel; Santosa, Andrew E.; Căciulescu, Alexandru Răzvan; Roychoudhury, Abhik (September 2021). "Smart Greybox Fuzzing". IEEE Transactions on Software Engineering 47 (9): 1980–1997. doi:10.1109/TSE.2019.2941681. ISSN 1939-3520. https://ieeexplore.ieee.org/document/8839290. 
  39. ↑ Böhme, Marcel; Pham, Van-Thuan; Nguyen, Manh-Dung; Roychoudhury, Abhik (2017-10-30). "Directed Greybox Fuzzing". Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. CCS '17 (New York, NY, USA: Association for Computing Machinery): 2329–2344. doi:10.1145/3133956.3134020. ISBN 978-1-4503-4946-8. https://doi.org/10.1145/3133956.3134020. 
  40. ↑ Poeplau, Sebastian; Francillon, Aurélien (2020) (in en). Symbolic execution with {SymCC}: Don't interpret, compile!. pp. 181–198. ISBN 978-1-939133-17-5. https://www.usenix.org/conference/usenixsecurity20/presentation/poeplau. 
  41. ↑ WinAFL, Google Project Zero, 2023-02-23, https://github.com/googleprojectzero/winafl, retrieved 2023-02-26 
  42. ↑ "Releases - AFLplusplus/AFLplusplus". https://github.com/AFLplusplus/AFLplusplus/releases. 
  43. ↑ Fioraldi, Andrea; Maier, Dominik; Eißfeldt, Heiko; Heuse, Marc (August 2020). "AFL++: Combining incremental steps of fuzzing research". 14th USENIX Workshop on Offensive Technologies (WOOT 20). https://www.usenix.org/conference/woot20/presentation/fioraldi. 
  44. ↑ "The AFL++ fuzzing framework" (in en). https://aflplus.plus/. 
  45. ↑ metzman, jonathan. "[afl++ Use AFL++ instead of AFL for fuzzing. by jonathanmetzman · Pull Request #5046 · google/oss-fuzz"] (in en). https://github.com/google/oss-fuzz/pull/5046. 
  46. ↑ Metzman et al. 2021, p. 1394.

Sources

Further reading

External links




Retrieved from "https://handwiki.org/wiki/index.php?title=Software:American_fuzzy_lop_(fuzzer)&oldid=3452333"