Cross-DSO control flow integrity (Clang part).

Clang-side cross-DSO CFI.

* Adds a command line flag -f[no-]sanitize-cfi-cross-dso.
* Links a runtime library when enabled.
* Emits __cfi_slowpath calls is bitset test fails.
* Emits extra hash-based bitsets for external CFI checks.
* Sets a module flag to enable __cfi_check generation during LTO.

This mode does not yet support diagnostics.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@255694 91177308-0d34-0410-b5e6-96231b3b80d8

docs/ControlFlowIntegrity.rst

@@ -24,10 +24,14 @@ You can also enable a subset of available :ref:`schemes <cfi-schemes>`.
 As currently implemented, all schemes rely on link-time optimization (LTO);
 so it is required to specify ``-flto``, and the linker used must support LTO,
 for example via the `gold plugin`_.
-To allow the checks to be implemented efficiently, the program must
-be structured such that certain object files are compiled with CFI enabled,
-and are statically linked into the program. This may preclude the use of
-shared libraries in some cases.
+
+To allow the checks to be implemented efficiently, the program must be
+structured such that certain object files are compiled with CFI
+enabled, and are statically linked into the program. This may preclude
+the use of shared libraries in some cases. Experimental support for
+:ref:`cross-DSO control flow integrity <cfi-cross-dso>` exists that
+does not have these requirements. This cross-DSO support has unstable
+ABI at this time.
 
 .. _gold plugin: http://llvm.org/docs/GoldPlugin.html
 
@@ -245,6 +249,16 @@ are typically defined outside of the linked program.
     # Ignore all types with a uuid attribute.
     type:attr:uuid
 
+.. _cfi-cross-dso:
+
+Shared library support
+======================
+
+Use **-f[no-]sanitize-cfi-cross-dso** to enable the cross-DSO control
+flow integrity mode, which allows all CFI schemes listed above to
+apply across DSO boundaries. As in the regular CFI, each DSO must be
+built with ``-flto``.
+
 Design
 ======
 

docs/ControlFlowIntegrityDesign.rst

@@ -366,3 +366,134 @@ Because the addresses of ``f``, ``g``, ``h`` are evenly spaced at a power of
 we can normally apply the `Alignment`_ and `Eliminating Bit Vector Checks
 for All-Ones Bit Vectors`_ optimizations thus simplifying the check at each
 call site to a range and alignment check.
+
+Shared library support
+======================
+
+**EXPERIMENTAL**
+
+The basic CFI mode described above assumes that the application is a
+monolithic binary; at least that all possible virtual/indirect call
+targets and the entire class hierarchy are known at link time. The
+cross-DSO mode, enabled with **-f[no-]sanitize-cfi-cross-dso** relaxes
+this requirement by allowing virtual and indirect calls to cross the
+DSO boundary.
+
+Assuming the following setup: the binary consists of several
+instrumented and several uninstrumented DSOs. Some of them may be
+dlopen-ed/dlclose-d periodically, even frequently.
+
+  - Calls made from uninstrumented DSOs are not checked and just work.
+  - Calls inside any instrumented DSO are fully protected.
+  - Calls between different instrumented DSOs are also protected, with
+     a performance penalty (in addition to the monolithic CFI
+     overhead).
+  - Calls from an instrumented DSO to an uninstrumented one are
+     unchecked and just work, with performance penalty.
+  - Calls from an instrumented DSO outside of any known DSO are
+     detected as CFI violations.
+
+In the monolithic scheme a call site is instrumented as
+
+.. code-block:: none
+
+   if (!InlinedFastCheck(f))
+     abort();
+   call *f
+
+In the cross-DSO scheme it becomes
+
+.. code-block:: none
+
+   if (!InlinedFastCheck(f))
+     __cfi_slowpath(CallSiteTypeId, f);
+   call *f
+
+CallSiteTypeId
+--------------
+
+``CallSiteTypeId`` is a stable process-wide identifier of the
+call-site type. For a virtual call site, the type in question is the class
+type; for an indirect function call it is the function signature. The
+mapping from a type to an identifier is an ABI detail. In the current,
+experimental, implementation the identifier of type T is calculated as
+follows:
+
+  -  Obtain the mangled name for "typeinfo name for T".
+  -  Calculate MD5 hash of the name as a string.
+  -  Reinterpret the first 8 bytes of the hash as a little-endian
+     64-bit integer.
+
+It is possible, but unlikely, that collisions in the
+``CallSiteTypeId`` hashing will result in weaker CFI checks that would
+still be conservatively correct.
+
+CFI_Check
+---------
+
+In the general case, only the target DSO knows whether the call to
+function ``f`` with type ``CallSiteTypeId`` is valid or not.  To
+export this information, every DSO implements
+
+.. code-block:: none
+
+   void __cfi_check(uint64 CallSiteTypeId, void *TargetAddr)
+
+This function provides external modules with access to CFI checks for
+the targets inside this DSO.  For each known ``CallSiteTypeId``, this
+functions performs an ``llvm.bitset.test`` with the corresponding bit
+set. It aborts if the type is unknown, or if the check fails.
+
+The basic implementation is a large switch statement over all values
+of CallSiteTypeId supported by this DSO, and each case is similar to
+the InlinedFastCheck() in the basic CFI mode.
+
+CFI Shadow
+----------
+
+To route CFI checks to the target DSO's __cfi_check function, a
+mapping from possible virtual / indirect call targets to
+the corresponding __cfi_check functions is maintained. This mapping is
+implemented as a sparse array of 2 bytes for every possible page (4096
+bytes) of memory. The table is kept readonly (FIXME: not yet) most of
+the time.
+
+There are 3 types of shadow values:
+
+  -  Address in a CFI-instrumented DSO.
+  -  Unchecked address (a “trusted” non-instrumented DSO). Encoded as
+     value 0xFFFF.
+  -  Invalid address (everything else). Encoded as value 0.
+
+For a CFI-instrumented DSO, a shadow value encodes the address of the
+__cfi_check function for all call targets in the corresponding memory
+page. If Addr is the target address, and V is the shadow value, then
+the address of __cfi_check is calculated as
+
+.. code-block:: none
+
+  __cfi_check = AlignUpTo(Addr, 4096) - (V + 1) * 4096
+
+This works as long as __cfi_check is aligned by 4096 bytes and located
+below any call targets in its DSO, but not more than 256MB apart from
+them.
+
+CFI_SlowPath
+------------
+
+The slow path check is implemented in compiler-rt library as
+
+.. code-block:: none
+
+  void __cfi_slowpath(uint64 CallSiteTypeId, void *TargetAddr)
+
+This functions loads a shadow value for ``TargetAddr``, finds the
+address of __cfi_check as described above and calls that.
+
+Position-independent executable requirement
+-------------------------------------------
+
+Cross-DSO CFI mode requires that the main executable is built as PIE.
+In non-PIE executables the address of an external function (taken from
+the main executable) is the address of that function’s PLT record in
+the main executable. This would break the CFI checks.

docs/UsersManual.rst

@@ -1042,6 +1042,12 @@ are listed below.
 
    Deprecated alias for ``-fsanitize-trap=undefined``.
 
+.. option:: -fsanitize-cfi-cross-dso
+
+   Enable cross-DSO control flow integrity checks. This flag modifies
+   the behavior of sanitizers in the ``cfi`` group to allow checking
+   of cross-DSO virtual and indirect calls.
+
 .. option:: -fno-assume-sane-operator-new
 
    Don't assume that the C++'s new operator is sane.

include/clang/Driver/Options.td

@@ -620,6 +620,12 @@ def fno_sanitize_undefined_trap_on_error : Flag<["-"], "fno-sanitize-undefined-t
                                            Group<f_clang_Group>;
 def fsanitize_link_cxx_runtime : Flag<["-"], "fsanitize-link-c++-runtime">,
                                  Group<f_clang_Group>;
+def fsanitize_cfi_cross_dso : Flag<["-"], "fsanitize-cfi-cross-dso">,
+                              Group<f_clang_Group>, Flags<[CC1Option]>,
+                              HelpText<"Enable control flow integrity (CFI) checks for cross-DSO calls.">;
+def fno_sanitize_cfi_cross_dso : Flag<["-"], "fno-sanitize-cfi-cross-dso">,
+                                 Group<f_clang_Group>, Flags<[CC1Option]>,
+                                 HelpText<"Disable control flow integrity (CFI) checks for cross-DSO calls.">;
 def funsafe_math_optimizations : Flag<["-"], "funsafe-math-optimizations">,
   Group<f_Group>;
 def fno_unsafe_math_optimizations : Flag<["-"], "fno-unsafe-math-optimizations">,

include/clang/Driver/SanitizerArgs.h

@@ -31,6 +31,7 @@ class SanitizerArgs {
   int CoverageFeatures;
   int MsanTrackOrigins;
   bool MsanUseAfterDtor;
+  bool CfiCrossDso;
   int AsanFieldPadding;
   bool AsanSharedRuntime;
   bool LinkCXXRuntimes;
@@ -53,6 +54,7 @@ class SanitizerArgs {
   bool needsSafeStackRt() const {
     return Sanitizers.has(SanitizerKind::SafeStack);
   }
+  bool needsCfiRt() const;
 
   bool requiresPIE() const;
   bool needsUnwindTables() const;

include/clang/Frontend/CodeGenOptions.def

@@ -120,6 +120,7 @@ CODEGENOPT(SanitizeMemoryTrackOrigins, 2, 0) ///< Enable tracking origins in
                                              ///< MemorySanitizer
 CODEGENOPT(SanitizeMemoryUseAfterDtor, 1, 0) ///< Enable use-after-delete detection
                                              ///< in MemorySanitizer
+CODEGENOPT(SanitizeCfiCrossDso, 1, 0) ///< Enable cross-dso support in CFI.
 CODEGENOPT(SanitizeCoverageType, 2, 0) ///< Type of sanitizer coverage
                                        ///< instrumentation.
 CODEGENOPT(SanitizeCoverageIndirectCalls, 1, 0) ///< Enable sanitizer coverage

lib/CodeGen/CGClass.cpp

@@ -2552,15 +2552,22 @@ void CodeGenFunction::EmitVTablePtrCheck(const CXXRecordDecl *RD,
 
   SanitizerScope SanScope(this);
 
-  llvm::Value *BitSetName = llvm::MetadataAsValue::get(
-      getLLVMContext(),
-      CGM.CreateMetadataIdentifierForType(QualType(RD->getTypeForDecl(), 0)));
+  llvm::Metadata *MD =
+      CGM.CreateMetadataIdentifierForType(QualType(RD->getTypeForDecl(), 0));
+  llvm::Value *BitSetName = llvm::MetadataAsValue::get(getLLVMContext(), MD);
 
   llvm::Value *CastedVTable = Builder.CreateBitCast(VTable, Int8PtrTy);
   llvm::Value *BitSetTest =
       Builder.CreateCall(CGM.getIntrinsic(llvm::Intrinsic::bitset_test),
                          {CastedVTable, BitSetName});
 
+  if (CGM.getCodeGenOpts().SanitizeCfiCrossDso) {
+    if (auto TypeId = CGM.CreateCfiIdForTypeMetadata(MD)) {
+      EmitCfiSlowPathCheck(BitSetTest, TypeId, CastedVTable);
+      return;
+    }
+  }
+
   SanitizerMask M;
   switch (TCK) {
   case CFITCK_VCall:
@@ -2578,9 +2585,9 @@ void CodeGenFunction::EmitVTablePtrCheck(const CXXRecordDecl *RD,
   }
 
   llvm::Constant *StaticData[] = {
-    EmitCheckSourceLocation(Loc),
-    EmitCheckTypeDescriptor(QualType(RD->getTypeForDecl(), 0)),
-    llvm::ConstantInt::get(Int8Ty, TCK),
+      EmitCheckSourceLocation(Loc),
+      EmitCheckTypeDescriptor(QualType(RD->getTypeForDecl(), 0)),
+      llvm::ConstantInt::get(Int8Ty, TCK),
   };
   EmitCheck(std::make_pair(BitSetTest, M), "cfi_bad_type", StaticData,
             CastedVTable);

lib/CodeGen/CGExpr.cpp

@@ -2532,6 +2532,34 @@ void CodeGenFunction::EmitCheck(
   EmitBlock(Cont);
 }
 
+void CodeGenFunction::EmitCfiSlowPathCheck(llvm::Value *Cond,
+                                           llvm::ConstantInt *TypeId,
+                                           llvm::Value *Ptr) {
+  auto &Ctx = getLLVMContext();
+  llvm::BasicBlock *Cont = createBasicBlock("cfi.cont");
+
+  llvm::BasicBlock *CheckBB = createBasicBlock("cfi.slowpath");
+  llvm::BranchInst *BI = Builder.CreateCondBr(Cond, Cont, CheckBB);
+
+  llvm::MDBuilder MDHelper(getLLVMContext());
+  llvm::MDNode *Node = MDHelper.createBranchWeights((1U << 20) - 1, 1);
+  BI->setMetadata(llvm::LLVMContext::MD_prof, Node);
+
+  EmitBlock(CheckBB);
+
+  llvm::Constant *SlowPathFn = CGM.getModule().getOrInsertFunction(
+      "__cfi_slowpath",
+      llvm::FunctionType::get(
+          llvm::Type::getVoidTy(Ctx),
+          {llvm::Type::getInt64Ty(Ctx),
+           llvm::PointerType::getUnqual(llvm::Type::getInt8Ty(Ctx))},
+          false));
+  llvm::CallInst *CheckCall = Builder.CreateCall(SlowPathFn, {TypeId, Ptr});
+  CheckCall->setDoesNotThrow();
+
+  EmitBlock(Cont);
+}
+
 void CodeGenFunction::EmitTrapCheck(llvm::Value *Checked) {
   llvm::BasicBlock *Cont = createBasicBlock("cont");
 
@@ -3823,21 +3851,25 @@ RValue CodeGenFunction::EmitCall(QualType CalleeType, llvm::Value *Callee,
       (!TargetDecl || !isa<FunctionDecl>(TargetDecl))) {
     SanitizerScope SanScope(this);
 
-    llvm::Value *BitSetName = llvm::MetadataAsValue::get(
-        getLLVMContext(),
-        CGM.CreateMetadataIdentifierForType(QualType(FnType, 0)));
+    llvm::Metadata *MD = CGM.CreateMetadataIdentifierForType(QualType(FnType, 0));
+    llvm::Value *BitSetName = llvm::MetadataAsValue::get(getLLVMContext(), MD);
 
     llvm::Value *CastedCallee = Builder.CreateBitCast(Callee, Int8PtrTy);
     llvm::Value *BitSetTest =
         Builder.CreateCall(CGM.getIntrinsic(llvm::Intrinsic::bitset_test),
                            {CastedCallee, BitSetName});
 
-    llvm::Constant *StaticData[] = {
-      EmitCheckSourceLocation(E->getLocStart()),
-      EmitCheckTypeDescriptor(QualType(FnType, 0)),
-    };
-    EmitCheck(std::make_pair(BitSetTest, SanitizerKind::CFIICall),
-              "cfi_bad_icall", StaticData, CastedCallee);
+    auto TypeId = CGM.CreateCfiIdForTypeMetadata(MD);
+    if (CGM.getCodeGenOpts().SanitizeCfiCrossDso && TypeId) {
+      EmitCfiSlowPathCheck(BitSetTest, TypeId, CastedCallee);
+    } else {
+      llvm::Constant *StaticData[] = {
+          EmitCheckSourceLocation(E->getLocStart()),
+          EmitCheckTypeDescriptor(QualType(FnType, 0)),
+      };
+      EmitCheck(std::make_pair(BitSetTest, SanitizerKind::CFIICall),
+                "cfi_bad_icall", StaticData, CastedCallee);
+    }
   }
 
   CallArgList Args;

lib/CodeGen/CGVTables.cpp

@@ -934,6 +934,7 @@ void CodeGenModule::EmitVTableBitSetEntries(llvm::GlobalVariable *VTable,
   llvm::NamedMDNode *BitsetsMD =
       getModule().getOrInsertNamedMetadata("llvm.bitsets");
   for (auto BitsetEntry : BitsetEntries)
-    BitsetsMD->addOperand(CreateVTableBitSetEntry(
-        VTable, PointerWidth * BitsetEntry.second, BitsetEntry.first));
+    CreateVTableBitSetEntry(BitsetsMD, VTable,
+                            PointerWidth * BitsetEntry.second,
+                            BitsetEntry.first);
 }

lib/CodeGen/CodeGenFunction.h

@@ -3009,6 +3009,11 @@ public:
                  StringRef CheckName, ArrayRef<llvm::Constant *> StaticArgs,
                  ArrayRef<llvm::Value *> DynamicArgs);
 
+  /// \brief Emit a slow path cross-DSO CFI check which calls __cfi_slowpath
+  /// if Cond if false.
+  void EmitCfiSlowPathCheck(llvm::Value *Cond, llvm::ConstantInt *TypeId,
+                            llvm::Value *Ptr);
+
   /// \brief Create a basic block that will call the trap intrinsic, and emit a
   /// conditional branch to it, for the -ftrapv checks.
   void EmitTrapCheck(llvm::Value *Checked);

lib/CodeGen/CodeGenModule.cpp

@@ -53,6 +53,7 @@
 #include "llvm/ProfileData/InstrProfReader.h"
 #include "llvm/Support/ConvertUTF.h"
 #include "llvm/Support/ErrorHandling.h"
+#include "llvm/Support/MD5.h"
 
 using namespace clang;
 using namespace CodeGen;
@@ -439,6 +440,11 @@ void CodeGenModule::Release() {
     getModule().addModuleFlag(llvm::Module::Error, "min_enum_size", EnumWidth);
   }
 
+  if (CodeGenOpts.SanitizeCfiCrossDso) {
+    // Indicate that we want cross-DSO control flow integrity checks.
+    getModule().addModuleFlag(llvm::Module::Override, "Cross-DSO CFI", 1);
+  }
+
   if (uint32_t PLevel = Context.getLangOpts().PICLevel) {
     llvm::PICLevel::Level PL = llvm::PICLevel::Default;
     switch (PLevel) {
@@ -736,6 +742,21 @@ void CodeGenModule::setFunctionDLLStorageClass(GlobalDecl GD, llvm::Function *F)
     F->setDLLStorageClass(llvm::GlobalVariable::DefaultStorageClass);
 }
 
+llvm::ConstantInt *
+CodeGenModule::CreateCfiIdForTypeMetadata(llvm::Metadata *MD) {
+  llvm::MDString *MDS = dyn_cast<llvm::MDString>(MD);
+  if (!MDS) return nullptr;
+
+  llvm::MD5 md5;
+  llvm::MD5::MD5Result result;
+  md5.update(MDS->getString());
+  md5.final(result);
+  uint64_t id = 0;
+  for (int i = 0; i < 8; ++i)
+    id |= static_cast<uint64_t>(result[i]) << (i * 8);
+  return llvm::ConstantInt::get(Int64Ty, id);
+}
+
 void CodeGenModule::setFunctionDefinitionAttributes(const FunctionDecl *D,
                                                     llvm::Function *F) {
   setNonAliasAttributes(D, F);
@@ -928,6 +949,49 @@ static void setLinkageAndVisibilityForGV(llvm::GlobalValue *GV,
   }
 }
 
+void CodeGenModule::CreateFunctionBitSetEntry(const FunctionDecl *FD,
+                                              llvm::Function *F) {
+  // Only if we are checking indirect calls.
+  if (!LangOpts.Sanitize.has(SanitizerKind::CFIICall))
+    return;
+
+  // Non-static class methods are handled via vtable pointer checks elsewhere.
+  if (isa<CXXMethodDecl>(FD) && !cast<CXXMethodDecl>(FD)->isStatic())
+    return;
+
+  // Additionally, if building with cross-DSO support...
+  if (CodeGenOpts.SanitizeCfiCrossDso) {
+    // Don't emit entries for function declarations. In cross-DSO mode these are
+    // handled with better precision at run time.
+    if (!FD->hasBody())
+      return;
+    // Skip available_externally functions. They won't be codegen'ed in the
+    // current module anyway.
+    if (getContext().GetGVALinkageForFunction(FD) == GVA_AvailableExternally)
+      return;
+  }
+
+  llvm::NamedMDNode *BitsetsMD =
+      getModule().getOrInsertNamedMetadata("llvm.bitsets");
+
+  llvm::Metadata *MD = CreateMetadataIdentifierForType(FD->getType());
+  llvm::Metadata *BitsetOps[] = {
+      MD, llvm::ConstantAsMetadata::get(F),
+      llvm::ConstantAsMetadata::get(llvm::ConstantInt::get(Int64Ty, 0))};
+  BitsetsMD->addOperand(llvm::MDTuple::get(getLLVMContext(), BitsetOps));
+
+  // Emit a hash-based bit set entry for cross-DSO calls.
+  if (CodeGenOpts.SanitizeCfiCrossDso) {
+    if (auto TypeId = CreateCfiIdForTypeMetadata(MD)) {
+      llvm::Metadata *BitsetOps2[] = {
+          llvm::ConstantAsMetadata::get(TypeId),
+          llvm::ConstantAsMetadata::get(F),
+          llvm::ConstantAsMetadata::get(llvm::ConstantInt::get(Int64Ty, 0))};
+      BitsetsMD->addOperand(llvm::MDTuple::get(getLLVMContext(), BitsetOps2));
+    }
+  }
+}
+
 void CodeGenModule::SetFunctionAttributes(GlobalDecl GD, llvm::Function *F,
                                           bool IsIncompleteFunction,
                                           bool IsThunk) {
@@ -970,19 +1034,7 @@ void CodeGenModule::SetFunctionAttributes(GlobalDecl GD, llvm::Function *F,
     F->addAttribute(llvm::AttributeSet::FunctionIndex,
                     llvm::Attribute::NoBuiltin);
 
-  // If we are checking indirect calls and this is not a non-static member
-  // function, emit a bit set entry for the function type.
-  if (LangOpts.Sanitize.has(SanitizerKind::CFIICall) &&
-      !(isa<CXXMethodDecl>(FD) && !cast<CXXMethodDecl>(FD)->isStatic())) {
-    llvm::NamedMDNode *BitsetsMD =
-        getModule().getOrInsertNamedMetadata("llvm.bitsets");
-
-    llvm::Metadata *BitsetOps[] = {
-        CreateMetadataIdentifierForType(FD->getType()),
-        llvm::ConstantAsMetadata::get(F),
-        llvm::ConstantAsMetadata::get(llvm::ConstantInt::get(Int64Ty, 0))};
-    BitsetsMD->addOperand(llvm::MDTuple::get(getLLVMContext(), BitsetOps));
-  }
+  CreateFunctionBitSetEntry(FD, F);
 }
 
 void CodeGenModule::addUsedGlobal(llvm::GlobalValue *GV) {
@@ -3874,14 +3926,28 @@ llvm::Metadata *CodeGenModule::CreateMetadataIdentifierForType(QualType T) {
   return InternalId;
 }
 
-llvm::MDTuple *CodeGenModule::CreateVTableBitSetEntry(
-    llvm::GlobalVariable *VTable, CharUnits Offset, const CXXRecordDecl *RD) {
+void CodeGenModule::CreateVTableBitSetEntry(llvm::NamedMDNode *BitsetsMD,
+                                            llvm::GlobalVariable *VTable,
+                                            CharUnits Offset,
+                                            const CXXRecordDecl *RD) {
+  llvm::Metadata *MD =
+      CreateMetadataIdentifierForType(QualType(RD->getTypeForDecl(), 0));
   llvm::Metadata *BitsetOps[] = {
-      CreateMetadataIdentifierForType(QualType(RD->getTypeForDecl(), 0)),
-      llvm::ConstantAsMetadata::get(VTable),
+      MD, llvm::ConstantAsMetadata::get(VTable),
       llvm::ConstantAsMetadata::get(
           llvm::ConstantInt::get(Int64Ty, Offset.getQuantity()))};
-  return llvm::MDTuple::get(getLLVMContext(), BitsetOps);
+  BitsetsMD->addOperand(llvm::MDTuple::get(getLLVMContext(), BitsetOps));
+
+  if (CodeGenOpts.SanitizeCfiCrossDso) {
+    if (auto TypeId = CreateCfiIdForTypeMetadata(MD)) {
+      llvm::Metadata *BitsetOps2[] = {
+          llvm::ConstantAsMetadata::get(TypeId),
+          llvm::ConstantAsMetadata::get(VTable),
+          llvm::ConstantAsMetadata::get(
+              llvm::ConstantInt::get(Int64Ty, Offset.getQuantity()))};
+      BitsetsMD->addOperand(llvm::MDTuple::get(getLLVMContext(), BitsetOps2));
+    }
+  }
 }
 
 // Fills in the supplied string map with the set of target features for the

lib/CodeGen/CodeGenModule.h

@@ -1106,15 +1106,21 @@ public:
   void EmitVTableBitSetEntries(llvm::GlobalVariable *VTable,
                                const VTableLayout &VTLayout);
 
+  /// Generate a cross-DSO type identifier for type.
+  llvm::ConstantInt *CreateCfiIdForTypeMetadata(llvm::Metadata *MD);
+
   /// Create a metadata identifier for the given type. This may either be an
   /// MDString (for external identifiers) or a distinct unnamed MDNode (for
   /// internal identifiers).
   llvm::Metadata *CreateMetadataIdentifierForType(QualType T);
 
-  /// Create a bitset entry for the given vtable.
-  llvm::MDTuple *CreateVTableBitSetEntry(llvm::GlobalVariable *VTable,
-                                         CharUnits Offset,
-                                         const CXXRecordDecl *RD);
+  /// Create a bitset entry for the given function and add it to BitsetsMD.
+  void CreateFunctionBitSetEntry(const FunctionDecl *FD, llvm::Function *F);
+
+  /// Create a bitset entry for the given vtable and add it to BitsetsMD.
+  void CreateVTableBitSetEntry(llvm::NamedMDNode *BitsetsMD,
+                               llvm::GlobalVariable *VTable, CharUnits Offset,
+                               const CXXRecordDecl *RD);
 
   /// \breif Get the declaration of std::terminate for the platform.
   llvm::Constant *getTerminateFn();

lib/CodeGen/MicrosoftCXXABI.cpp

@@ -1523,15 +1523,14 @@ void MicrosoftCXXABI::emitVTableBitSetEntries(VPtrInfo *Info,
 
   if (Info->PathToBaseWithVPtr.empty()) {
     if (!CGM.IsCFIBlacklistedRecord(RD))
-      BitsetsMD->addOperand(
-          CGM.CreateVTableBitSetEntry(VTable, AddressPoint, RD));
+      CGM.CreateVTableBitSetEntry(BitsetsMD, VTable, AddressPoint, RD);
     return;
   }
 
   // Add a bitset entry for the least derived base belonging to this vftable.
   if (!CGM.IsCFIBlacklistedRecord(Info->PathToBaseWithVPtr.back()))
-    BitsetsMD->addOperand(CGM.CreateVTableBitSetEntry(
-        VTable, AddressPoint, Info->PathToBaseWithVPtr.back()));
+    CGM.CreateVTableBitSetEntry(BitsetsMD, VTable, AddressPoint,
+                                Info->PathToBaseWithVPtr.back());
 
   // Add a bitset entry for each derived class that is laid out at the same
   // offset as the least derived base.
@@ -1550,14 +1549,12 @@ void MicrosoftCXXABI::emitVTableBitSetEntries(VPtrInfo *Info,
     if (!Offset.isZero())
       return;
     if (!CGM.IsCFIBlacklistedRecord(DerivedRD))
-      BitsetsMD->addOperand(
-          CGM.CreateVTableBitSetEntry(VTable, AddressPoint, DerivedRD));
+      CGM.CreateVTableBitSetEntry(BitsetsMD, VTable, AddressPoint, DerivedRD);
   }
 
   // Finally do the same for the most derived class.
   if (Info->FullOffsetInMDC.isZero() && !CGM.IsCFIBlacklistedRecord(RD))
-    BitsetsMD->addOperand(
-        CGM.CreateVTableBitSetEntry(VTable, AddressPoint, RD));
+    CGM.CreateVTableBitSetEntry(BitsetsMD, VTable, AddressPoint, RD);
 }
 
 void MicrosoftCXXABI::emitVTableDefinitions(CodeGenVTables &CGVT,

lib/Driver/SanitizerArgs.cpp

@@ -160,7 +160,12 @@ bool SanitizerArgs::needsUbsanRt() const {
   return (Sanitizers.Mask & NeedsUbsanRt & ~TrapSanitizers.Mask) &&
          !Sanitizers.has(Address) &&
          !Sanitizers.has(Memory) &&
-         !Sanitizers.has(Thread);
+         !Sanitizers.has(Thread) &&
+         !CfiCrossDso;
+}
+
+bool SanitizerArgs::needsCfiRt() const {
+  return CfiCrossDso;
 }
 
 bool SanitizerArgs::requiresPIE() const {
@@ -184,6 +189,7 @@ void SanitizerArgs::clear() {
   AsanFieldPadding = 0;
   AsanSharedRuntime = false;
   LinkCXXRuntimes = false;
+  CfiCrossDso = false;
 }
 
 SanitizerArgs::SanitizerArgs(const ToolChain &TC,
@@ -430,6 +436,14 @@ SanitizerArgs::SanitizerArgs(const ToolChain &TC,
                  TC.getTriple().getArch() == llvm::Triple::x86_64);
   }
 
+  if (AllAddedKinds & CFI) {
+    CfiCrossDso = Args.hasFlag(options::OPT_fsanitize_cfi_cross_dso,
+                               options::OPT_fno_sanitize_cfi_cross_dso, false);
+    // Without PIE, external function address may resolve to a PLT record, which
+    // can not be verified by the target module.
+    NeedPIE |= CfiCrossDso;
+  }
+
   // Parse -f(no-)?sanitize-coverage flags if coverage is supported by the
   // enabled sanitizers.
   if (AllAddedKinds & SupportsCoverage) {
@@ -580,6 +594,9 @@ void SanitizerArgs::addArgs(const ToolChain &TC, const llvm::opt::ArgList &Args,
   if (MsanUseAfterDtor)
     CmdArgs.push_back(Args.MakeArgString("-fsanitize-memory-use-after-dtor"));
 
+  if (CfiCrossDso)
+    CmdArgs.push_back(Args.MakeArgString("-fsanitize-cfi-cross-dso"));
+
   if (AsanFieldPadding)
     CmdArgs.push_back(Args.MakeArgString("-fsanitize-address-field-padding=" +
                                          llvm::utostr(AsanFieldPadding)));

lib/Driver/Tools.cpp

@@ -2792,6 +2792,8 @@ collectSanitizerRuntimes(const ToolChain &TC, const ArgList &Args,
   }
   if (SanArgs.needsSafeStackRt())
     StaticRuntimes.push_back("safestack");
+  if (SanArgs.needsCfiRt())
+    StaticRuntimes.push_back("cfi");
 }
 
 // Should be called before we add system libraries (C++ ABI, libstdc++/libc++,

lib/Frontend/CompilerInvocation.cpp

@@ -586,6 +586,7 @@ static bool ParseCodeGenArgs(CodeGenOptions &Opts, ArgList &Args, InputKind IK,
       getLastArgIntValue(Args, OPT_fsanitize_memory_track_origins_EQ, 0, Diags);
   Opts.SanitizeMemoryUseAfterDtor =
       Args.hasArg(OPT_fsanitize_memory_use_after_dtor);
+  Opts.SanitizeCfiCrossDso = Args.hasArg(OPT_fsanitize_cfi_cross_dso);
   Opts.SSPBufferSize =
       getLastArgIntValue(Args, OPT_stack_protector_buffer_size, 8, Diags);
   Opts.StackRealignment = Args.hasArg(OPT_mstackrealign);

test/CodeGen/cfi-icall-cross-dso.c

@@ -0,0 +1,49 @@
+// RUN: %clang_cc1 -triple x86_64-unknown-linux -O1 -fsanitize=cfi-icall -fsanitize-cfi-cross-dso -emit-llvm -o - %s | FileCheck --check-prefix=CHECK --check-prefix=ITANIUM %s
+// RUN: %clang_cc1 -triple x86_64-pc-windows-msvc -O1 -fsanitize=cfi-icall  -fsanitize-cfi-cross-dso -emit-llvm -o - %s | FileCheck --check-prefix=CHECK --check-prefix=MS %s
+
+void caller(void (*f)()) {
+  f();
+}
+
+static void g(void) {}
+void h(void);
+
+typedef void (*Fn)(void);
+Fn g1() {
+  return &g;
+}
+Fn h1() {
+  return &h;
+}
+
+inline void foo() {}
+void bar() { foo(); }
+
+// ITANIUM: call i1 @llvm.bitset.test(i8* %{{.*}}, metadata !"_ZTSFvE"), !nosanitize
+// ITANIUM: call void @__cfi_slowpath(i64 6588678392271548388, i8* %{{.*}}) {{.*}}, !nosanitize
+
+// MS: call i1 @llvm.bitset.test(i8* %{{.*}}, metadata !"?6AX@Z"), !nosanitize
+// MS: call void @__cfi_slowpath(i64 4195979634929632483, i8* %{{.*}}) {{.*}}, !nosanitize
+
+// ITANIUM: define available_externally void @foo()
+// MS: define linkonce_odr void @foo()
+
+// Check that we emit both string and hash based bit set entries for static void g(),
+// and don't emit them for the declaration of h().
+
+// CHECK-NOT: !{!"{{.*}}", void ()* @h, i64 0}
+// CHECK: !{!"{{.*}}", void ()* @g, i64 0}
+// CHECK-NOT: !{!"{{.*}}", void ()* @h, i64 0}
+// CHECK: !{i64 {{.*}}, void ()* @g, i64 0}
+// CHECK-NOT: !{!"{{.*}}", void ()* @h, i64 0}
+
+// ITANIUM-NOT: !{!{{.*}}, void ()* @foo,
+// ITANIUM: !{!"_ZTSFvE", void ()* @bar, i64 0}
+// ITANIUM-NOT: !{!{{.*}}, void ()* @foo,
+// ITANIUM: !{i64 6588678392271548388, void ()* @bar, i64 0}
+// ITANIUM-NOT: !{!{{.*}}, void ()* @foo,
+
+// MS: !{!"?6AX@Z", void ()* @foo, i64 0}
+// MS: !{i64 4195979634929632483, void ()* @foo, i64 0}
+
+// CHECK: !{i32 4, !"Cross-DSO CFI", i32 1}

test/CodeGenCXX/cfi-cross-dso.cpp

@@ -0,0 +1,45 @@
+// RUN: %clang_cc1 -triple x86_64-unknown-linux -fsanitize=cfi-vcall -fsanitize-cfi-cross-dso -emit-llvm -o - %s | FileCheck --check-prefix=CHECK --check-prefix=ITANIUM %s
+// RUN: %clang_cc1 -triple x86_64-pc-windows-msvc -fsanitize=cfi-vcall  -fsanitize-cfi-cross-dso -emit-llvm -o - %s | FileCheck --check-prefix=CHECK --check-prefix=MS %s
+
+struct A {
+  A();
+  virtual void f();
+};
+
+A::A() {}
+void A::f() {}
+
+void caller(A* a) {
+  a->f();
+}
+
+namespace {
+struct B {
+  virtual void f();
+};
+
+void B::f() {}
+} // namespace
+
+void g() {
+  B b;
+  b.f();
+}
+
+// MS: @[[B_VTABLE:.*]] = private unnamed_addr constant [2 x i8*] {{.*}}@"\01??_R4B@?A@@6B@"{{.*}}@"\01?f@B@?A@@UEAAXXZ"
+
+// CHECK:   %[[VT:.*]] = load void (%struct.A*)**, void (%struct.A*)***
+// CHECK:   %[[VT2:.*]] = bitcast {{.*}}%[[VT]] to i8*, !nosanitize
+// ITANIUM:   %[[TEST:.*]] = call i1 @llvm.bitset.test(i8* %[[VT2]], metadata !"_ZTS1A"), !nosanitize
+// MS:   %[[TEST:.*]] = call i1 @llvm.bitset.test(i8* %[[VT2]], metadata !"?AUA@@"), !nosanitize
+// CHECK:   br i1 %[[TEST]], label %[[CONT:.*]], label %[[SLOW:.*]], {{.*}} !nosanitize
+// CHECK: [[SLOW]]:
+// ITANIUM:   call void @__cfi_slowpath(i64 7004155349499253778, i8* %[[VT2]]) {{.*}} !nosanitize
+// MS:   call void @__cfi_slowpath(i64 -8005289897957287421, i8* %[[VT2]]) {{.*}} !nosanitize
+// CHECK:   br label %[[CONT]], !nosanitize
+// CHECK: [[CONT]]:
+// CHECK:   call void %{{.*}}(%struct.A* %{{.*}})
+
+// No hash-based bit set entry for (anonymous namespace)::B
+// ITANIUM-NOT: !{i64 {{.*}}, [3 x i8*]* @_ZTVN12_GLOBAL__N_11BE,
+// MS-NOT: !{i64 {{.*}}, [2 x i8*]* @[[B_VTABLE]],

test/Driver/fsanitize.c

@@ -263,6 +263,15 @@
 // CHECK-CFI-NOTRAP-WIN: -emit-llvm-bc
 // CHECK-CFI-NOTRAP-WIN-NOT: -fsanitize-trap=cfi
 
+// RUN: %clang -target x86_64-linux-gnu -fsanitize=cfi -fsanitize-cfi-cross-dso -flto -c %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-CFI-CROSS-DSO
+// RUN: %clang -target x86_64-linux-gnu -fsanitize=cfi -flto -c %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-CFI-NO-CROSS-DSO
+// RUN: %clang -target x86_64-linux-gnu -fsanitize=cfi -fsanitize-cfi-cross-dso -fno-sanitize-cfi-cross-dso -flto -c %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-CFI-NO-CROSS-DSO
+// RUN: %clang -target x86_64-linux-gnu -fsanitize=cfi -fno-sanitize-cfi-cross-dso -fsanitize-cfi-cross-dso -flto -c %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-CFI-CROSS-DSO
+// CHECK-CFI-CROSS-DSO: -emit-llvm-bc
+// CHECK-CFI-CROSS-DSO: -fsanitize-cfi-cross-dso
+// CHECK-CFI-NO-CROSS-DSO: -emit-llvm-bc
+// CHECK-CFI-NO-CROSS-DSO-NOT: -fsanitize-cfi-cross-dso
+
 // RUN: %clang_cl -fsanitize=address -c -MDd -### -- %s 2>&1 | FileCheck %s -check-prefix=CHECK-ASAN-DEBUGRTL
 // RUN: %clang_cl -fsanitize=address -c -MTd -### -- %s 2>&1 | FileCheck %s -check-prefix=CHECK-ASAN-DEBUGRTL
 // RUN: %clang_cl -fsanitize=address -c -LDd -### -- %s 2>&1 | FileCheck %s -check-prefix=CHECK-ASAN-DEBUGRTL

test/Driver/sanitizer-ld.c

@@ -291,6 +291,46 @@
 // CHECK-LSAN-ASAN-LINUX: libclang_rt.asan-x86_64
 // CHECK-LSAN-ASAN-LINUX-NOT: libclang_rt.lsan
 
+// CFI by itself does not link runtime libraries.
+// RUN: %clang -fsanitize=cfi %s -### -o %t.o 2>&1 \
+// RUN:     -target x86_64-unknown-linux \
+// RUN:     --sysroot=%S/Inputs/basic_linux_tree \
+// RUN:   | FileCheck --check-prefix=CHECK-CFI-LINUX %s
+// CHECK-CFI-LINUX: "{{.*}}ld{{(.exe)?}}"
+// CHECK-CFI-LINUX-NOT: libclang_rt.
+
+// CFI with diagnostics links the UBSan runtime.
+// RUN: %clang -fsanitize=cfi -fno-sanitize-trap=cfi -fsanitize-recover=cfi \
+// RUN:     %s -### -o %t.o 2>&1\
+// RUN:     -target x86_64-unknown-linux \
+// RUN:     --sysroot=%S/Inputs/basic_linux_tree \
+// RUN:   | FileCheck --check-prefix=CHECK-CFI-DIAG-LINUX %s
+// CHECK-CFI-DIAG-LINUX: "{{.*}}ld{{(.exe)?}}"
+// CHECK-CFI-CROSS-DSO-LINUX-NOT: libclang_rt.
+// CHECK-CFI-DIAG-LINUX: libclang_rt.ubsan
+// CHECK-CFI-CROSS-DSO-LINUX-NOT: libclang_rt.
+
+// Cross-DSO CFI links the CFI runtime.
+// RUN: %clang -fsanitize=cfi -fsanitize-cfi-cross-dso %s -### -o %t.o 2>&1 \
+// RUN:     -target x86_64-unknown-linux \
+// RUN:     --sysroot=%S/Inputs/basic_linux_tree \
+// RUN:   | FileCheck --check-prefix=CHECK-CFI-CROSS-DSO-LINUX %s
+// CHECK-CFI-CROSS-DSO-LINUX: "{{.*}}ld{{(.exe)?}}"
+// CHECK-CFI-CROSS-DSO-LINUX-NOT: libclang_rt.
+// CHECK-CFI-CROSS-DSO-LINUX: libclang_rt.cfi
+// CHECK-CFI-CROSS-DSO-LINUX-NOT: libclang_rt.
+
+// Cross-DSO CFI with diagnostics links just the CFI runtime.
+// RUN: %clang -fsanitize=cfi -fsanitize-cfi-cross-dso %s -### -o %t.o 2>&1 \
+// RUN:     -fno-sanitize-trap=cfi -fsanitize-recover=cfi \
+// RUN:     -target x86_64-unknown-linux \
+// RUN:     --sysroot=%S/Inputs/basic_linux_tree \
+// RUN:   | FileCheck --check-prefix=CHECK-CFI-CROSS-DSO-DIAG-LINUX %s
+// CHECK-CFI-CROSS-DSO-DIAG-LINUX: "{{.*}}ld{{(.exe)?}}"
+// CHECK-CFI-CROSS-DSO-DIAG-LINUX-NOT: libclang_rt.
+// CHECK-CFI-CROSS-DSO-DIAG-LINUX: libclang_rt.cfi
+// CHECK-CFI-CROSS-DSO-DIAG-LINUX-NOT: libclang_rt.
+
 // RUN: %clangxx -fsanitize=address %s -### -o %t.o 2>&1 \
 // RUN:     -mmacosx-version-min=10.6 \
 // RUN:     -target x86_64-apple-darwin13.4.0 \