Foreign function interface

Short description: Interface to call functions from other programming languages

A foreign function interface (FFI) is a mechanism by which a program written in one programming language can call routines or make use of services written or compiled in another one. An FFI is often used in contexts where calls are made into binary dynamic-link library.

Naming

The term comes from the specification for Common Lisp, which explicitly refers to the programming language feature enabling for inter-language calls as such; the term is also often used officially by the Haskell, Rust, PHP, Python, and LuaJIT (Lua) interpreter and compiler documentations. Other languages use other terminology: the Ada programming language talks about "language bindings", while Java refers to its FFI as the JNI (Java Native Interface) or JNA (Java Native Access). Foreign function interface has become generic terminology for mechanisms which provide such services.

Operation

The primary function of a foreign function interface is to mate the semantics and calling conventions of one programming language (the host language, or the language which defines the FFI), with the semantics and conventions of another (the guest language). This process must also take into consideration the runtime environments and application binary interfaces of both. This can be done in several ways:

• Requiring that guest-language functions which are to be host-language callable be specified or implemented in a particular way, often using a compatibility library of some sort.
• Use of a tool to automatically "wrap" guest-language functions with appropriate glue code, which performs any necessary translation.
• Use of wrapper libraries
• Restricting the set of host language capabilities which can be used cross-language. For example, C++ functions called from C may not (in general) include reference parameters or throw exceptions.

FFIs may be complicated by the following considerations:

• If one language supports garbage collection (GC) and the other does not; care must be taken that the non-GC language code does nothing to cause GC in the other to fail. In JNI, for example, C code which "holds on to" object references that it receives from Java must "register" this fact with the Java runtime environment (JRE); otherwise, Java may delete objects before C has finished with them. (The C code must also explicitly release its link to any such object once C has no further need of that object.)
• Complicated or non-trivial objects or datatypes may be difficult to map from one environment to another.
• It may not be possible for both languages to maintain references to the same instance of a mutable object, due to the mapping issue above.
• One or both languages may be running on a virtual machine (VM); moreover, if both are, these will probably be different VMs.
• Cross-language inheritance and other differences, such as between type systems or between object-composition models, may be especially difficult.

Examples of FFIs include:

• Ada language bindings, allowing not only to call foreign functions but also to export its functions and methods to be called from non-Ada code.
• C++ has a trivial FFI with C, as the languages share a significant common subset. The primary effect of the extern "C" declaration in C++ is to disable C++ name mangling. With other languages, separate utils or middleware are used, examples include:
  • GNOME project: GObject Introspection
  • SWIG
  • Chromium project: Blink and V8 engine use an Interface description language (IDL) compiler for standard JavaScript interfaces
  • Others IDL compilers
• Clean provides a bidirectional FFI with all languages following C or the stdcall calling convention.
• Common Lisp
• CNI, alternative to JNI used in the GNU compiler environment.
• One of the bases of the Component Object Model is a common interface format, which natively uses the same types as Visual Basic for strings and arrays.
• D does it the same way as C++ does, with extern "C" through extern (C++)
• Dart includes dart:ffi library to call native C code for mobile, command-line, and server applications
• Dynamic languages, such as Python, Perl, Tcl, and Ruby, all provide easy access to native code written in C/C++ (or any other language obeying C/C++ calling conventions).
• Factor has FFIs for C, Fortran, Objective-C, and Windows COM; all of these enable importing and calling arbitrary shared libraries dynamically.
• Fortran 2003 has a module ISO_C_BINDING which provides interoperable data types (both intrinsic types and POD structs), interoperable pointers, interoperable global data stores, and mechanisms for calling C from Fortran and for calling Fortran from C. It has been improved in the Fortran 2018 standard.
• Go can call C code directly via the "C" pseudo-package.
• GWT, in which Java is compiled to JavaScript, has an FFI called JSNI which allows Java source to call arbitrary JavaScript functions, and for JavaScript to call back into Java.
• Haskell
• JNI, which provides an interface between Java and C/C++, the preferred systems languages on most systems where Java is deployed. JNA provides an interface with native libraries without having to write glue code. Another example is JNR
• LuaJIT, a just-in-time implementation of Lua, has an FFI that allows "calling external C functions and using C data structures from pure Lua code".
• Nim has an FFI which enables it to use source from C, C++, and Objective-C. It can also interface with Javascript.
• JavaScript usually runs inside web browser runtimes that don't provide direct access to system libraries or commands to run, but there are few exceptions:
  • Node.js provides functions to open precompiled .node modules that in turn may provide access to non-builtin resources.
  • Deno, provides kind of FFI interface via dlopen(...) functions.
  • Bun provides a built-in module, bun:ffi, to efficiently call native libraries directly from JavaScript.
• Julia has ccall keyword to call C (and other languages, e.g. Fortran); while packages, providing similar no-boilerplate support, are available for some languages e.g. for Python (to e.g. provide OO support and GC support), Java (and supports other JDK-languages, such as Scala) and R. Interactive use with C++ is also possible with Cxx.jl package.
• PhoneGap (was called by the name Apache Callback, but now Apache Cordova) is a platform for building native mobile applications using HTML, CSS and JavaScript. Additionally has FFIs via JavaScript callback functions for access to methods and properties of mobile phone's native features including Accelerometer, Camera (also PhotoLibrary and SavedPhotoAlbum), Compass, Storage (SQL database and localStorage), Notification, Media and Capture (playing and recording or audio and video), File, Contacts (address book), Events, Device and Connection information.[1],[2].
• PHP provides FFI to C.
• Python provides the ctypes and cffi modules. For example, the ctypes module can load C functions from shared libraries/DLLs on-the-fly and translate simple data types automatically between Python and C semantics as follows:

  import ctypes

libc = ctypes.CDLL('/lib/libc.so.6') # Under Linux/Unix t = libc.time(None) # Equivalent C code: t = time(NULL)

print(t)

• P/Invoke, which provides an interface between the Microsoft Common Language Runtime and native code.
• Racket has a native FFI based heavily on macros that enables importing arbitrary shared libraries dynamically.
• Raku can call Ruby, Python, Perl, Brainfuck, Lua, C, C++, Go, Scheme Guile/Gambit and Rust
• Ruby provides FFI either through the ffi gem, or through the standard library fiddle.

  require 'fiddle'

libm = Fiddle.dlopen('/lib/libm.so.6')

1. Equivalent to: double floor(double x);

floor = Fiddle::Function.new(

 libm.sym('floor'),     # ptr is a referenced function(, or symbol), of a Fiddle::Handle.
 [Fiddle::TYPE_DOUBLE], # args is an Array of arguments, passed to the ptr function.
 Fiddle::TYPE_DOUBLE    # ret_type is the return type of the function

)

1. Equivalent to: floor(3.14159);

floor.call(3.14159) #=> 3.0

• Rust defines a foreign function interface to functions with various standard ABIs. There is also a library for interfacing with Elixir, Rustler.
• Visual Basic has a declarative syntax that allows it to call non-Unicode C functions.
• Wolfram Language provides a technology called WSTP (Wolfram Symbolic Transfer Protocol) which enables bidirectional calling of code between other languages with bindings for C++, Java, .NET and other languages.
• Zig provides FFI to c using the builtin cImport function.

In addition, many FFIs can be generated automatically: for example, SWIG. However, in the case of an extension language a semantic inversion of the relationship of guest and host can occur, when a smaller body of extension language is the guest invoking services in the larger body of host language, such as writing a small plugin for GIMP.

Some FFIs are restricted to free standing functions, while others also allow calls of functions embedded in an object or class (often called method calls); some even permit migration of complex datatypes or objects across the language boundary.

In most cases, an FFI is defined by a "higher-level" language, so that it may employ services defined and implemented in a lower level language, typically a systems language like C or C++. This is typically done to either access OS services in the language in which the OS' API is defined, or for performance considerations.

Many FFIs also provide the means for the called language to invoke services in the host language as well.

The term foreign function interface is generally not used to describe multi-lingual runtimes such as the Microsoft Common Language Runtime, where a common "substrate" is provided which enables any CLR-compliant language to use services defined in any other. (However, in this case the CLR does include an FFI, P/Invoke, to call outside the runtime.) In addition, many distributed computing architectures such as the Java remote method invocation (RMI), RPC, CORBA, SOAP and D-Bus permit different services to be written in different languages; such architectures are generally not considered FFIs.

Special cases

There are some special cases, in which the languages compile into the same bytecode VM, like Clojure and Java, as well as Elixir and Erlang. Since there is no interface, it is not an FFI, strictly speaking, while it offers the same functionality to the user.

See also

• Language interoperability
• Interface definition language
• Calling convention
• Name mangling
• Application programming interface
• Application binary interface
• Comparison of application virtual machines
• SWIG
• Remote procedure call
• libffi

External links

• c2.com: Foreign function interface
• Haskell 98 Foreign Function Interface
• Allegro Common Lisp FFI
• A Foreign Function Interface generator for occam-pi
• UFFI: Lisp Universal Foreign Function Interface
• CFFI: Common Foreign Function Interface, for Common Lisp
• Java Native Interface: Programmer's Guide and Specification
• The JNI Specification
• JSNI (JavaScript Native Interface)
• dyncall library using assembly call kernels for a variety of processors,OS and calling conventions
• FFCALL
• C/Invoke
• libffi