Luke Imhoff
luke_imhoff@rapid7.com | Kronic.Deth@gmail.com | |
@limhoff-r7 | @KronicDeth | |
@KronicDeth |
Loader | Extension Required | Loads Only Once | Synchrony | Loads |
---|---|---|---|---|
Kernel.load |
Yes | No | Synchronous | File |
Kernel.require |
No | Yes | Synchronous | File |
Kernel.autoload |
No | Yes | Asynchronous | File |
ActiveSupport::Autoload#autoload |
No | Yes | Asynchronous | File |
ActiveSupport::Dependencies#autoload_paths |
No | Either | Asynchronous | Directory Tree |
A standard Rails application
daemon UI uses to run metasploit-framework
610 lines
config.autoload_paths
require_relative
in prosvc
require
s in
prosvc
autoload_paths
between UI and
prosvc
ack
config.autoload_paths
outside Rails
config.autoload_paths
in
prosvc
config.autoload_paths
works
config.autoload_paths
config.autoload_paths
is mentioned in the Configuring guide
set_autoload_paths
?
Gem | Purpose |
---|---|
actionmailer | sending mail |
activerecord | database records |
activeresource | non-database records |
bundler | dependency management |
actionpack | ? |
activesupport | ? |
railties | ? |
cd ~/git
mkdir rails
cd rails
git clone git@github.com:rails/rails.git
cd rails
git checkout v3.2.2
ack set_autoload_paths
ack set_autoload_path
railties/lib/rails/engine.rb:536
_all_autoload_paths
set_autoload_paths
config.autoload_paths
config.eager_load_paths
config.autoload_once_paths
ActiveSupport::Dependencies.autoload_paths.unshift(config.autoload_paths)
Rails::Engine
set_load_path
_all_load_paths
System | Initializers | Configuration |
---|---|---|
Rails |
Rails::Engine
|
Rails::Engine::Configuration
|
prosvc |
Metasploit::Configured
|
Metasploit::Configuration
|
require 'active_support/concern'
# TODO this should be a Rails::Engine if engine is ever moved under ui
module Metasploit::Configured
extend ActiveSupport::Concern
module ClassMethods
def configuration
@configuration ||= self::Configuration.new
end
def initialize!
configuration.initializers.each do |initializer|
send(initializer)
end
end
def eager_load!
configuration.dependencies.each(&:eager_load!)
# Adapted from Rails::Engine#eager_load!
configuration.eager_load_paths.each do |load_path|
# strip the load_path itself from the name because the load_path is already add to $LOAD_PATH by
# {#set_load_path}.
# strip extension as it's not normally passed to require.
require_path_regex = /\A#{Regexp.escape(load_path)}\/(.*)\.rb\Z/
glob = "#{load_path}/**/*.rb"
Dir.glob(glob) do |path|
require_path = path.sub(require_path_regex, '\1')
require_dependency require_path
end
end
end
def root
configuration.root
end
def set_load_path
configuration.dependencies.each(&:set_load_path)
# reverse because unshift is pushing onto the front
configuration.all_autoload_paths.reverse_each do |path|
if File.directory?(path)
$LOAD_PATH.unshift(path)
end
# uniq at end in case all_autoload_paths include paths already in $LOAD_PATH
$LOAD_PATH.uniq!
end
end
def set_autoload_paths
configuration.dependencies.each(&:set_autoload_paths)
ActiveSupport::Dependencies.autoload_paths.unshift(*configuration.all_autoload_paths)
# Freeze so future modifications error out instead of being silently ignored
configuration.autoload_paths.freeze
configuration.eager_load_paths.freeze
end
end
end
require 'active_support/concern'
module Metasploit::Configured
module ClassMethods
def configuration
@configuration ||= self::Configuration.new
end
end
end
self::Configuration
|
Configuration
|
---|---|
Scoped to base class | Scoped to Metasploit::Configured::ClassMethods |
module Metasploit::Configured
module ClassMethods
def initialize!
configuration.initializers.each do |initializer|
send(initializer)
end
end
end
end
initialize!
to match Rails convention
# Load the rails application
require File.expand_path('../application', __FILE__)
# Initialize the rails application
Pro::Application.initialize!
module Metasploit::Configured
module ClassMethods
def eager_load!
configuration.dependencies.each(&:eager_load!)
# Adapted from Rails::Engine#eager_load!
configuration.eager_load_paths.each do |load_path|
# strip the load_path itself from the name because the load_path is
# already add to $LOAD_PATH by {#set_load_path}.
# strip extension as it's not normally passed to require.
require_path_regex = /\A#{Regexp.escape(load_path)}\/(.*)\.rb\Z/
glob = "#{load_path}/**/*.rb"
Dir.glob(glob) do |path|
require_path = path.sub(require_path_regex, '\1')
require_dependency require_path
end
end
end
end
end
module Metasploit::Configured
module ClassMethods
def root
configuration.root
end
end
end
module Metasploit::Configured
module ClassMethods
def set_load_path
configuration.dependencies.each(&:set_load_path)
# reverse because unshift is pushing onto the front
configuration.all_autoload_paths.reverse_each do |path|
if File.directory?(path)
$LOAD_PATH.unshift(path)
end
# uniq at end in case all_autoload_paths include paths
# already in $LOAD_PATH
$LOAD_PATH.uniq!
end
end
end
end
module Rails
class Engine < Railtie
def ordered_railties
railties.all + [self]
end
def initializers
initializers = []
ordered_railties.each do |r|
if r == self
initializers += super
else
initializers += r.initializers
end
end
initializers
end
end
end
module Metasploit::Configured
module ClassMethods
def set_load_path
configuration.dependencies.each(&:set_load_path)
# reverse because unshift is pushing onto the front
configuration.all_autoload_paths.reverse_each do |path|
if File.directory?(path)
$LOAD_PATH.unshift(path)
end
# uniq at end in case all_autoload_paths include paths
# already in $LOAD_PATH
$LOAD_PATH.uniq!
end
end
end
end
module Rails
class Engine < Railtie
initializer :set_load_path, :before => :bootstrap_hook do
_all_load_paths.reverse_each do |path|
$LOAD_PATH.unshift(path) if
File.directory?(path)
end
$LOAD_PATH.uniq!
end
end
end
module Metasploit
class Configuration
def all_autoload_paths
all_autoload_paths = autoload_paths + eager_load_paths
unique_autoload_paths = all_autoload_paths.uniq
unique_autoload_paths
end
def autoload_paths
@autoload_paths ||= []
end
def dependencies
@dependencies ||= []
end
def eager_load_paths
@eager_load_paths ||= []
end
def initializers
# XXX not sure I like that initializer methods that only exist in
# Metasploit::Configured are used here.
@initializers ||= [
:set_load_path,
:set_autoload_paths,
:eager_load!
]
end
attr_reader :root
end
end
module Metasploit
class Configuration
def initializers
# XXX not sure I like that initializer methods that only exist in
# Metasploit::Configured are used here.
@initializers ||= [
:set_load_path,
:set_autoload_paths,
:eager_load!
]
end
end
end
$:.unshift(File.expand_path(File.join(File.dirname(__FILE__), 'lib'))) # engine/lib
$:.unshift(File.expand_path(File.join(File.dirname(__FILE__), '..', 'msf3', 'lib'))) # msf3/lib
# ...
require_relative '../ui/config/initializers/carrierwave' # ui/config/initializers
#
# Load specific patches for system libraries
#
if arch == "win32"
require "win32/registry" # engine/lib
require "patches/win32_registry" # engine/lib
end
# ...
#
# Require our basic libraries and start loading
#
require 'metasploit_data_models' # (gem)
include MetasploitDataModels
# Figure out the rails path
rails_app_path = File.expand_path(File.dirname(__FILE__)) + "/../ui/app"
# Bring all Mdm::* models into view
Dir.glob("#{rails_app_path}/models/mdm/*.rb").each do |mdm_model_path| # ui/app/models
require mdm_model_path
end
# ---- BEGIN - Load all things SocialEngineering from Rails ----
module LiquidTemplating; end
liquid_files = Dir.glob("#{rails_app_path}/../lib/liquid_templating/*.rb") # ui/lib
liquid_files.sort.each do |liquid_path|
require liquid_path
end
# TODO: any better way to have both Rails and prosvc happy w/ talking to License?
require "#{rails_app_path}/models/license.rb" # ui/app/models
# Declare the module manually - Rails does this from directory structure
module SocialEngineering; end
require "#{rails_app_path}/uploaders/social_engineering/campaign_file_uploader.rb" # ui/app/uploaders
manually_loaded_models = ["human_target", "email", "web_page_attack_config_interface"]
skippable_files = ["campaign_task"]
require "#{rails_app_path}/models/social_engineering/human_target.rb" # ui/app/models
# avoid chicken-egg
require "#{rails_app_path}/models/social_engineering/web_page_attack_config_interface.rb" # ui/app/models
# avoid chicken-egg
se_files = Dir.glob("#{rails_app_path}/models/social_engineering/*.rb")
se_files.sort.each do |se_model_path|
file_name = File.basename(se_model_path).split('.').first
next if skippable_files.include? file_name
next if manually_loaded_models.include? file_name
require se_model_path # ui/app/models
end
# ---- END - Load all things SocialEngineering from Rails ----
require 'msf/ui' # msf3/lib
require 'rex' # msf3/lib
# Metasploit Core API
require 'msf/core/rpc/v10/service' # msf3/lib
# Pro API
require 'pro/filters' # engine/lib
require 'pro/config' # engine/lib
require 'pro/rpc/v10/rpc_pro' # engine/lib
# Pro Mixins
require 'pro/mixins' # engine/lib
# Pro Hooks
require 'pro/hooks' # engine/lib
# Pro Client
require 'pro/client' # engine/lib
# Background Daemon
require 'pro/bgdaemon' # engine/lib
# NginX
require "pro/nginx" # engine/lib
require 'logger' # (standard library)
# ...
# Load individual namespaces and models
require_relative "#{rails_app_path}/models/user_session.rb" # ui/app/models
require_relative "#{rails_app_path}/models/task_chain.rb" # ui/app/models
require_relative "#{rails_app_path}/models/scheduled_task.rb" # ui/app/models
require_relative "#{rails_app_path}/models/campaign.rb" # ui/app/models
require_relative "#{rails_app_path}/models/attachment.rb" # ui/app/models
require_relative "#{rails_app_path}/models/email_template.rb" # ui/app/models
require_relative "#{rails_app_path}/models/email_address.rb" # ui/app/models
require_relative "#{rails_app_path}/models/web_template.rb" # ui/app/models
# Load classes in lib root
ui_lib_path = File.expand_path(File.dirname(__FILE__)) + "/../ui/lib"
require "#{ui_lib_path}/sender.rb" # ui/lib
lib_files = Dir.glob("#{ui_lib_path}/*.rb")
lib_files.sort.each do |lib_class|
require lib_class # ui/lib
end
engine/lib
msf3/lib
ui/config/initializers
ui/app/models
ui/lib
ui/app/uploaders
module Metasploit::Framework
class Configuration < Metasploit::Configuration
def initialize
@root = Metasploit::Pro.root.join('msf3')
lib_pathname = root.join('lib')
lib_path = lib_pathname.to_s
autoload_paths << lib_path
base_pathname = root.join('lib', 'base')
base_path = base_pathname.to_s
autoload_paths << base_path
end
end
end
module Metasploit::Pro::Engine
class Configuration < Metasploit::Configuration
def initialize
@dependencies = [
Metasploit::Framework,
Metasploit::Pro::UI
]
@root = Metasploit::Pro.root.join('engine')
lib_pathname = root.join('lib')
lib_path = lib_pathname.to_s
autoload_paths << lib_path
end
end
end
module Metasploit::Pro::UI
class Configuration < Metasploit::Configuration
def initialize
@root = Metasploit::Pro.root.join('ui')
lib_pathname = root.join('lib')
lib_path = lib_pathname.to_s
autoload_paths << lib_path
# Some models reference the controllers,
# so app/controllers needs to be added to autoload paths
controllers_pathname = root.join('app', 'controllers')
controllers_path = controllers_pathname.to_s
autoload_paths << controllers_path
uploaders_pathname = root.join('app', 'uploaders')
uploaders_path = uploaders_pathname.to_s
autoload_paths << uploaders_path
models_pathname = root.join('app', 'models')
models_path = models_pathname.to_s
autoload_paths << models_path
end
end
end
require_relative
in prosvc
require
s in
prosvc
autoload_paths
between UI and
prosvc
modules
directorymsfconsole
use
command can instantiate a Metasploit Moduleset
command configures the instance of the Metasploit Modulerun
launch the instance of the Metasploit Modulemsfpro
msfconsole
with access to Metasploit Pro featuresrequire_relative
in prosvc
require
s in
prosvc
autoload_paths
between UI and prosvc
name
name.presence
klass_name
klass_name.to_s
klass_name.to_s.scan
nesting
Module#name
Module#name
containing
::
Metasploit3?
MetasploitN
N
is Metasploit3
Ruby Classes/ModulesMetasploit4
Ruby Classes/Modulesrequire_relative
in prosvc
require
s in
prosvc
autoload_paths
between UI and prosvc
Module.new
usages
Module.new
usages
Module.new
UsagesMsf::ModuleManager#reload_module
Msf::ModuleManager#load_module_from_file
require_relative
in prosvc
require
s in
prosvc
autoload_paths
between UI and prosvc
Module.new
usages
Module.new
usages
Module.new
in Msf::ModuleManager#load_module_from_file
Module.new
in Msf::ModuleManager#reload_module
Module.new
name
is
/
separated (
auxiliary/pro/social_engineering/web_phish
)
String#camelize
will convert
/
separated to
Module#name
>> "auxiliary/pro/social/engineering/web_phish".camelize
=> "Auxiliary::Pro::SocialEngineering::WebPhish"
Object
?Hex-escaping
child_name
not passed to Module.new
require_relative
in prosvc
require
s in
prosvc
autoload_paths
between UI and prosvc
Module.new
usages
Module.new
in Msf::ModuleManager#load_module_from_file
Module.new
in Msf::ModuleManager#reload_module
#cache_entries
,
#rebuild_cache
,
#refresh_cache
)
Msf::Module
instances (
#create
)
#demand_load_module
,
#failed
,
#has_module_file_changed?
,
#load_module_from_file
,
#load_module_source
,
#load_modules
,
#load_modules_from_directory
,
#on_module_load
)
#auxiliary
,
#encoders
,
#exploits
,
#init_module_set
,
#module_names
,
#module_set
,
#module_types
,
#nops
,
#payloads
,
#post
)
#add_module_path
,
#remove_module_path
)
#reload_module
,
#reload_modules
)
#add_module
,
#auto_subscribe_module
,
#register_type_extension
)
Msf::ModuleManager#add_module
#auto_subscribe_module
framework.events.on_module_load
Probably loading related...
Called from #on_module_load
, so definitely loading
Msf::ModuleManager#register_type_extension
Dead Code?
def
= deadinclude
Ruby Module in base Ruby Class/Module
module Msf
class ModuleManager
require 'msf/core/module_manager/cache'
include Msf::ModuleManager::Cachet
end
end
Internal | External | |
---|---|---|
Single | add_module | |
autosubscribe_module | ||
demand_load_module | ||
failed | ||
has_module_file_changed? | ||
load_module_from_file | ||
load_module_source | ||
on_module_load | ||
Multiple | load_modules | load_modules_from_directory |
Msf::ModuleSet#add_module
Msf::ModuleManager::Loading
#auto_subscribe_module
too due to usage#load_module_from_file
in directory category, so extract to Msf::Modules::Loader::Directory
Check Usages to confirm category
Msf::ModuleSet
framework.modules.demand_load_module
Keep in Msf::ModuleManager::Loading
modules_failed
attribute.Find usages to check for public interface
framework.modules.failed
, so part of public interfaceKeep in Msf::ModuleManager::Loading
on_*
implies callbackMsf::ModuelManager
or Msf::ModuleSet
stateKeep in Msf::ModuleManager::Loading
class Msf::Modules::Loader::Directory < Msf::Modules::Loader::Base
protected
# Yields the module_reference_name for each module file found under the directory path.
#
# @param [String] path The path to the directory.
# @yield (see Msf::Modules::Loader::Base#each_module_reference_name)
# @yieldparam [String] path The path to the directory.
# @yieldparam [String] type The type correlated with the directory under path.
# @yieldparam module_reference_name (see Msf::Modules::Loader::Base#each_module_reference_name)
# @return (see Msf::Modules::Loader::Base#each_module_reference_name)
def each_module_reference_name(path)
::Dir.foreach(path) do |entry|
if entry.downcase == '.svn'
next
end
full_entry_path = ::File.join(path, entry)
type = entry.singularize
unless ::File.directory?(full_entry_path) and
module_manager.type_enabled? type
next
end
full_entry_pathname = Pathname.new(full_entry_path)
# Try to load modules from all the files in the supplied path
Rex::Find.find(full_entry_path) do |entry_descendant_path|
if module_path?(entry_descendant_path)
entry_descendant_pathname = Pathname.new(entry_descendant_path)
relative_entry_descendant_pathname = entry_descendant_pathname.relative_path_from(full_entry_pathname)
relative_entry_descendant_path = relative_entry_descendant_pathname.to_path
# The module_reference_name doesn't have a file extension
module_reference_name = module_reference_name_from_path(relative_entry_descendant_path)
yield path, type, module_reference_name
end
end
end
end
end
.rb
filesMsf::Modules::Loader::Base#load_module
#module_path
#read_module_content
#namespace_module_transaction
# Returns the full path to the module file on disk.
#
# @param (See Msf::Modules::Loader::Base#module_path
# @return [String] Path toe the module file on disk.
def module_path(parent_path, type, module_reference_name)
file_name = module_reference_name + MODULE_EXTENSION
type_directory = DIRECTORY_BY_TYPE[type]
full_path = File.join(parent_path, type_directory, file_name)
full_path
end
# Load the module content from the on disk file.
#
# @param (see Msf::Modules::Loader::Base#read_module_content)
# @return (see Msf::Modules::Loader::Base#read_module_content)
def read_module_content(parent_path, type, module_reference_name)
full_path = module_path(parent_path, type, module_refrence_name)
File.read(full_path)
end
#load_module
require_relative
in prosvc
require
s in
prosvc
autoload_paths
between UI and prosvc
Module.new
usages
Module.new
in Msf::ModuleManager#load_module_from_file
Module.new
in Msf::ModuleManager#reload_module
Congratulation! The test case works, but... does the test case cover all ways that all 1851 Metasploit Modules are actually written?
NOPE!
Module Creation | #create_module_namespace |
---|---|
Module Evaluation | #module_eval_with_lexical_scope |
Module#module_eval
work?Msf
from lexical scopeModule
or Class
module Msf::ModuleManager
puts Module.nesting.inspect # [Msf::ModuleManager]
end
module Msf
module ModuleManager
puts Module.nesting.inspect # [Msf::ModuleManager, Msf]
end
end
[Msf::ModuleManager, Msf]
[Msf::Modules::Loader::Base]
require_relative
in prosvc
require
s in
prosvc
autoload_paths
between UI and prosvc
Module.new
usages
Module.new
in Msf::ModuleManager#load_module_from_file
Module.new
in Msf::ModuleManager#reload_module
Msf
Msf
namespace, which is invalid for metasploit-framework (Metasploit::Framework
is correct)Module#module_eval
with block doesn't capture lexical scope; have to use String
require_relative
in prosvc
require
s in
prosvc
autoload_paths
between UI and prosvc
Module.new
usages
Module.new
in Msf::ModuleManager#load_module_from_file
Module.new
in Msf::ModuleManager#reload_module
Msf
Msf
in lexical scope using Module#module_eval(String)
wrapper_module (bugged)
namespace_module_names (fixed)
module_eval
require_relative
in prosvc
require
s in
prosvc
autoload_paths
between UI and prosvc
Module.new
usages
Module.new
in Msf::ModuleManager#load_module_from_file
Module.new
in Msf::ModuleManager#reload_module
Msf
in lexical scope using Module#module_eval(String)
Module.new
creates anonymous Ruby Modules
Module#name
when a constant is set to the Module
ActiveSupport::Dependencies
’s const_missing
Class
or Module
per fileinclude
d Module
sClass
hierarchiesmodule_eval
module_eval
captures the lexical scope only with Stringmodule
declaration has a different lexical scope than ::
separated namesModule.nesting
module_eval
allows debugging in string code.I’d like to thank James “Egypt” Lee for imparting his knowledge of how to use the various msfconsole commands to ensure my changes didn’t break anything, fixing my lambda vs proc bug, and Ruby 1.8-incompatibility. I’d like to thank Samuel “Shuckins” Huckins for testing these changes against Metasploit Pro. I’d like to thank HD Moore for spotting when I missed automatic namespace names colliding with real Ruby Modules. I’d like to thank Trevor Rosen for allowing me to spend weeks to fix this the right way. I’d like to thank Regina Imhoff and Sonny Gonzalez for reviewing the accompanying article. I'd like to thank the Metasploit team for reviewing this presentation.