Rippled Audit

int main(int argc, char** argv)

As described in the Build System documentation the primary rippled executable is compiled from a long list of sources, each satisfying a particular bit of functionality required by the service. The module which defines the main entry point into the application is src/ripple/app/main/Main.cpp. It is this logic that is executed first when the rippled executable is run (besides the boilerplate logic built in by the compiler).

761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
int main (int argc, char** argv)
{
#ifdef _MSC_VER
    {
        // Work around for https://svn.boost.org/trac/boost/ticket/10657
        // Reported against boost version 1.56.0.  If an application's
        // first call to GetTimeZoneInformation is from a coroutine, an
        // unhandled exception is generated.  A workaround is to call
        // GetTimeZoneInformation at least once before launching any
        // coroutines.  At the time of this writing the _ftime call is
        // used to initialize the timezone information.
        struct _timeb t;
    #ifdef _INC_TIME_INL
            _ftime_s (&t);
    #else
            _ftime (&t);
    #endif
    }
    ripple::sha512_deprecatedMSVCWorkaround();
#endif

We start off by invoking a few workarounds for the MSWIN platform. Besides the call to GetTimeZoneInformation we call sha512_deprecatedMSVCWorkaround defined in digest.h to instantiate encryption-related static definitions.

The next blocks of code is fairly self explanatory, a minimal GCC version is defined and checked and we see some optional debugging logic (commented out and conditional depending on compile definition)

782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
#if defined(__GNUC__) && !defined(__clang__)
    auto constexpr gccver = (__GNUC__ * 100 * 100) +
                            (__GNUC_MINOR__ * 100) +
                            __GNUC_PATCHLEVEL__;

    static_assert (gccver >= 50100,
        "GCC version 5.1.0 or later is required to compile rippled.");
#endif

    //
    // These debug heap calls do nothing in release or non Visual Studio builds.
    //

    // Checks the heap at every allocation and deallocation (slow).
    //
    //beast::Debug::setAlwaysCheckHeap (false);

    // Keeps freed memory blocks and fills them with a guard value.
    //
    //beast::Debug::setHeapDelayedFree (false);

    // At exit, reports all memory blocks which have not been freed.
    //
#if RIPPLE_DUMP_LEAKS_ON_EXIT
    beast::Debug::setHeapReportLeaks (true);
#else
    beast::Debug::setHeapReportLeaks (false);
#endif

Starting on line 811 we see the std::atexit C++ callback is invoked to register google::protobuf::ShutdownProtobufLibrary to be invoked on application exit (cleaning up the protocol buffers) and std::set_terminate is invoked to register ripple::terminateHandler (defined in ripple/core/impl/TerminateHandler.cpp) to log forcefully terminated threads.

811
812
813
    atexit(&google::protobuf::ShutdownProtobufLibrary);

    std::set_terminate(ripple::terminateHandler);

After this we invoke ripple::run with the command line arguments passed to the application (more on this briefly. This method will not return until after the application is finished, at which point we invoke beast::basic_seconds_clock_main_hook defined in src/ripple/beast/clock/basic_seconds_clock.h containing another workaround for the MSWIN platform (shutting down the internal seconds_clock thread)

815
816
817
818
819
820
    auto const result (ripple::run (argc, argv));

    beast::basic_seconds_clock_main_hook();

    return result;
}

ripple::run is defined above in the same source file.

299
300
301
302
303
int run (int argc, char** argv)
{
    using namespace std;

    beast::setCurrentThreadName ("rippled: main");

beast::setCurrentThreadName associates a user friendly name with the current executing thread for debugging purposes. After this we see the initialization of the command line option parser:

307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
    po::variables_map vm;

    std::string importText;
    {
        importText += "Import an existing node database (specified in the [";
        importText += ConfigSection::importNodeDatabase ();
        importText += "] configuration file section) into the current ";
        importText += "node database (specified in the [";
        importText += ConfigSection::nodeDatabase ();
        importText += "] configuration file section).";
    }
    std::string shardsText;
    {
        shardsText += "Validate an existing shard database (specified in the [";
        shardsText += ConfigSection::shardDatabase();
        shardsText += "] configuration file section).";
    }

    // Set up option parsing.
    //
    po::options_description gen ("General Options");
    gen.add_options ()
    ("conf", po::value<std::string> (), "Specify the configuration file.")
    ("debug", "Enable normally suppressed debug logging")
    ("fg", "Run in the foreground.")
    ("help,h", "Display this message.")
    ("quorum", po::value <std::size_t> (),
        "Override the minimum validation quorum.")
    ("silent", "No output to the console after startup.")
    ("standalone,a", "Run with no peers.")
    ("verbose,v", "Verbose logging.")
    ("version", "Display the build version.")
    ;

    po::options_description data ("Ledger/Data Options");
    data.add_options ()
    ("import", importText.c_str ())
    ("ledger", po::value<std::string> (),
        "Load the specified ledger and start from the value given.")
    ("ledgerfile", po::value<std::string> (), "Load the specified ledger file.")
    ("load", "Load the current ledger from the local DB.")
    ("net", "Get the initial ledger from the network.")
    ("nodetoshard", "Import node store into shards")
    ("replay","Replay a ledger close.")
    ("start", "Start from a fresh Ledger.")
    ("vacuum", po::value<std::string>(),
        "VACUUM the transaction db. Mandatory string argument specifies "
        "temporary directory path.")
    ("valid", "Consider the initial ledger a valid network ledger.")
    ("validateShards", shardsText.c_str ())
    ;

    po::options_description rpc ("RPC Client Options");
    rpc.add_options()
    ("rpc",
        "Perform rpc command - see below for available commands. "
        "This is assumed if any positional parameters are provided.")
    ("rpc_ip", po::value <std::string> (),
        "Specify the IP address for RPC command. "
        "Format: <ip-address>[':'<port-number>]")
    ("rpc_port", po::value <std::uint16_t> (),
        "DEPRECATED: include with rpc_ip instead. "
        "Specify the port number for RPC command.")
    ;

    po::options_description test ("Unit Test Options");
    test.add_options()
    ("quiet,q",
        "Suppress test suite messages, "
        "including suite/case name (at start) and test log messages.")
    ("unittest,u", po::value <std::string> ()->implicit_value (""),
        "Perform unit tests. The optional argument specifies one or "
        "more comma-separated selectors. Each selector specifies a suite name, "
        "full-name (lib.module.suite), module, or library "
        "(checked in that ""order).")
    ("unittest-arg", po::value <std::string> ()->implicit_value (""),
        "Supplies an argument string to unit tests. If provided, this argument "
        "is made available to each suite that runs. Interpretation of the "
        "argument is handled individually by any suite that accesses it -- "
        "as such, it typically only make sense to provide this when running "
        "a single suite.")
    ("unittest-ipv6", "Use IPv6 localhost when running unittests (default is IPv4).")
    ("unittest-log",
        "Force unit test log message output. Only useful in combination with "
        "--quiet, in which case log messages will print but suite/case names "
        "will not.")
    ("unittest-jobs", po::value <std::size_t> (),
        "Number of unittest jobs to run in parallel (child processes).")
    ;

    // These are hidden options, not intended to be shown in the usage/help message
    po::options_description hidden ("Hidden Options");
    hidden.add_options()
    ("parameters", po::value< vector<string> > (),
        "Specify rpc command and parameters. This option must be repeated "
        "for each command/param. Positional parameters also serve this purpose, "
        "so this option is not needed for users")
    ("unittest-child",
        "For internal use only when spawning child unit test processes.")
    ;

    // Interpret positional arguments as --parameters.
    po::positional_options_description p;
    p.add ("parameters", -1);

    po::options_description all;
    all.add(gen).add(rpc).add(data).add(test).add(hidden);

    po::options_description desc;
    desc.add(gen).add(rpc).add(data).add(test);

    // Parse options, if no error.
    try
    {
        po::store (po::command_line_parser (argc, argv)
            .options (all)                // Parse options.
            .positional (p)               // Remainder as --parameters.
            .run (),
            vm);
        po::notify (vm);                  // Invoke option notify functions.
    }
    catch (std::exception const&)
    {
        std::cerr << "rippled: Incorrect command line syntax." << std::endl;
        std::cerr << "Use '--help' for a list of options." << std::endl;
        return 1;
}

See rippled documentation for specifics on accepted command line options. After successful parsing, the help and version options are processed (printing out the requested information and then exiting):

433
434
435
436
437
438
439
440
441
442
443
444
    if (vm.count ("help"))
    {
        printHelp (desc);
        return 0;
    }

    if (vm.count ("version"))
    {
        std::cout << "rippled version " <<
            BuildInfo::getVersionString () << std::endl;
        return 0;
    }

If the user opted to run the unit tests, those are executed before returning (terminating the process):

449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
    if (vm.count ("unittest"))
    {
        std::string argument;

        if (vm.count("unittest-arg"))
            argument = vm["unittest-arg"].as<std::string>();

        std::size_t numJobs = 1;
        bool unittestChild = false;
        if (vm.count("unittest-jobs"))
            numJobs = std::max(numJobs, vm["unittest-jobs"].as<std::size_t>());
        unittestChild = bool (vm.count("unittest-child"));

        return runUnitTests(
            vm["unittest"].as<std::string>(), argument,
            bool (vm.count ("quiet")),
            bool (vm.count ("unittest-log")),
            unittestChild,
            bool (vm.count ("unittest-ipv6")),
            numJobs,
            argc,
            argv);
    }
    else
    {
        if (vm.count("unittest-jobs"))
        {
            // unittest jobs only makes sense with `unittest`
            std::cerr << "rippled: '--unittest-jobs' specified without '--unittest'.\n";
            std::cerr << "To run the unit tests the '--unittest' option must be present.\n";
            return 1;
        }
    }

Next the code checks to see if the user specified the vacuum command line option so as to perform a VACUUM operation on the local database.

499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
    if (vm.count("vacuum"))
    {
        DatabaseCon::Setup dbSetup = setup_DatabaseCon(*config);
        if (dbSetup.standAlone)
        {
            std::cerr << "vacuum not applicable in standalone mode.\n";
            return -1;
        }
        boost::filesystem::path dbPath = dbSetup.dataDir / TxnDBName;
        auto txnDB = std::make_unique<DatabaseCon> (dbSetup, TxnDBName,
            TxnDBInit, TxnDBCount);
        if (txnDB.get() == nullptr)
        {
            std::cerr << "Cannot create connection to " << dbPath.string() <<
                '\n';
            return -1;
        }
        boost::system::error_code ec;
        uintmax_t dbSize = boost::filesystem::file_size(dbPath, ec);
        if (ec)
        {
            std::cerr << "Error checking size of " << dbPath.string() << ": "
                << ec.message() << '\n';
            return -1;
        }

        assert(dbSize != static_cast<uintmax_t>(-1));

        std::string tmpDir = vm["vacuum"].as<std::string>();
        boost::filesystem::path tmpPath = tmpDir;
        if (boost::filesystem::space(tmpPath, ec).available < dbSize)
        {
            if (ec)
            {
                std::cerr << "Error checking status of " << tmpPath.string()
                    << ": " << ec.message() << '\n';
            }
            else
            {
                std::cerr << "A valid directory for vacuuming must be "
                             "specified on a filesystem with at least "
                             "as much free space as the size of " <<
                             dbPath.string() << ", which is " <<
                             dbSize << " bytes. The filesystem for " <<
                             tmpPath.string() << " only has " <<
                             boost::filesystem::space(tmpPath, ec).available
                             << " bytes.\n";
            }
            return -1;
        }

        auto db = txnDB->checkoutDb();
        std::uint32_t pageSize;
        try
        {
            *db << "PRAGMA page_size;", soci::into(pageSize);
            std::cout << "VACUUM beginning. page_size: " << pageSize
                << std::endl;
            *db << "PRAGMA journal_mode=OFF;";
            *db << "PRAGMA temp_store_directory=\"" << tmpPath.string()
                << "\";";
            *db << "VACUUM;";
            *db << "PRAGMA journal_mode=WAL;";
            *db << "PRAGMA page_size;", soci::into(pageSize);
        }
        catch (soci::soci_error const& e)
        {
            std::cerr << "SQLite error: " << e.what() << '\n';
            return 1;
        }
        std::cout << "VACUUM finished. page_size: " << pageSize << std::endl;

        return 0;
    }

Above we see a DB connection is established after which the VACUUM sql command is invoked. Various sanity checks are conducted to ensure the operating environment for the vacuum is available.

Next we set various config options based on command line parameters (the config data structure is defined in src/ripple/core/Config.h)

576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
    if (vm.count ("start"))
        config->START_UP = Config::FRESH;

    if (vm.count ("import"))
        config->doImport = true;

    if (vm.count("nodetoshard"))
        config->nodeToShard = true;

    if (vm.count ("validateShards"))
        config->validateShards = true;

    if (vm.count ("ledger"))
    {
        config->START_LEDGER = vm["ledger"].as<std::string> ();
        if (vm.count("replay"))
            config->START_UP = Config::REPLAY;
        else
            config->START_UP = Config::LOAD;
    }
    else if (vm.count ("ledgerfile"))
    {
        config->START_LEDGER = vm["ledgerfile"].as<std::string> ();
        config->START_UP = Config::LOAD_FILE;
    }
    else if (vm.count ("load"))
    {
        config->START_UP = Config::LOAD;
    }

    if (vm.count ("valid"))
    {
        config->START_VALID = true;
    }

    if (vm.count ("net"))
    {
        if ((config->START_UP == Config::LOAD) ||
            (config->START_UP == Config::REPLAY))
        {
            std::cerr <<
                "Net and load/reply options are incompatible" << std::endl;
            return -1;
        }

        config->START_UP = Config::NETWORK;
    }

    // Override the RPC destination IP address. This must
    // happen after the config file is loaded.
    if (vm.count ("rpc_ip"))
    {
        auto res = beast::IP::Endpoint::from_string_checked(
            vm["rpc_ip"].as<std::string>());
        if (! res.second)
        {
            std::cerr << "Invalid rpc_ip = " <<
                vm["rpc_ip"].as<std::string>() << "\n";
            return -1;
        }

        if (res.first.port() == 0)
        {
            std::cerr << "No port specified in rpc_ip.\n";
            if (vm.count ("rpc_port"))
            {
                std::cerr << "WARNING: using deprecated rpc_port param.\n";
                try
                {
                    res.first.at_port(vm["rpc_port"].as<std::uint16_t>());
                    if (res.first.port() == 0)
                        throw std::domain_error("0");
                }
                catch(std::exception const& e)
                {
                    std::cerr << "Invalid rpc_port = " << e.what() << "\n";
                    return -1;
                }
            }
            else
                return -1;
        }

        config->rpc_ip = std::move(res.first);
    }

    if (vm.count ("quorum"))
    {
        try
        {
            config->VALIDATION_QUORUM = vm["quorum"].as <std::size_t> ();
            if (config->VALIDATION_QUORUM == std::size_t{})
            {
                throw std::domain_error("0");
            }
        }
        catch(std::exception const& e)
        {
            std::cerr << "Invalid value specified for --quorum ("
                      << e.what() << ")\n";
            return -1;
        }
    }

Next the logs are initialized (the base Logs class is defined in src/ripple/basics/Log.h):

679
680
681
682
683
684
685
686
687
    using namespace beast::severities;
    Severity thresh = kInfo;

    if (vm.count ("quiet"))
        thresh = kFatal;
    else if (vm.count ("verbose"))
        thresh = kTrace;

    auto logs = std::make_unique<Logs>(thresh);

At this point one of two logic paths are followed:

  1. If no RPC command is specified via the command line, the web server is started and blocks until shutdown
  2. If a RPC command is specified, it is executed (connecting to a remote rippled server) and then the application exits immediately afterwards (more on this path below)

In the case of server execution:

693
694
695
696
        // We want at least 1024 file descriptors. We'll
        // tweak this further.
        if (!adjustDescriptorLimit(1024, logs->journal("Application")))
          return -1;

adjustDescriptorLimit (defined above) is invoked which in return calls setrlimit (if needed) to increase RLIMIT_NOFILE, the number of file descriptors which the process can create to 1024. If the limit is failed to be increased, the process will exit with an error.

The Sustain subsystem (defined in src/ripple/basics/impl/Sustain.cpp) is started after this and is responsible for monitoring the main process and restarting it on errors:

697
698
699
700
701
702
703
        if (HaveSustain() && !vm.count ("fg") && !config->standalone())
        {
            auto const ret = DoSustain ();

            if (!ret.empty ())
                std::cerr << "Watchdog: " << ret << std::endl;
        }

After this debug logging is configured (if applicable) and the centralized time management system is initialized (TimeKeeper).

Finally the main application class is initialized and the server is started, blocking until completion:

714
715
716
717
718
719
720
721
722
723
724
725
        auto app = make_Application(
            std::move(config),
            std::move(logs),
            std::move(timeKeeper));

        // ...some lines skipped for brevity...

        // Start the server
        app->doStart(true /*start timers*/);

        // Block until we get a stop RPC.
        app->run();

make_Application is defined in src/ripple/app/main/Application.h and implemented in src/ripple/app/main/Application.cpp, simple instantiating the internal ApplicationImp class, which in return wraps the Application interface. We will dive into the details of Application Submodule in the next section

In the case where an RPC command is specified via the rippled command line (see logic fork above), the application updates the thread name and invokes the command via the RPC subsytem

750
751
752
753
754
    beast::setCurrentThreadName ("rippled: rpc");
    return RPCCall::fromCommandLine (
        *config,
        vm["parameters"].as<std::vector<std::string>>(),
        *logs);
<<Back Home >>Next