Installing a package from the IDE

Download

In my VCL Component Installer, I needed to find a way to install the package I had created and/or opened in the IDE. This article describes how it can be done.

Building

Building the package is straightforward. It uses IOTAProject.GetProjectBuilder to get a IOTAProjectBuilder and then uses its BuildProject procedure to build the package. If this is successful, the installation starts.

Installing

Installing was by far the most tricky part. There is no Open Tools API (OTA) function for this, so good advice was needed. There was one successful approach on the web, but that meant emulating menu item clicks and button clicks in various menus and dialogs and programmatically selecting files from an open file dialog. Too flakey, and not what I wanted.

I first tried to use LoadPackage and GetProcAddress to find the Register procedure of the package. But packages don’t have a single Register procedure at all. The individual units each have their own. So I thought of using GetPackageInfo and IOTAPackageServices to get the info for each unit. The Register procedure of each unit is exposed as @Unitname@Register$qqrv, IOW as a C++ mangled name. I tried this with one unit in one package, but even though I called the proper Register function, nothing was installed.

So I decided to take another approach, and let the IDE do the dirty work for me. I first wanted to find out what functions and methods the IDE calls when it installs a package. I created a simple package with one unit with a Register procedure. In that Register procedure, I raised an Exception. When I installed the package, there was, as expected, an error when the exception was raised, and the IDE gave me a nice stack trace. Here is the relevant part:

[2147618A]{delphicoreide120.bpl} PasCppPakMgr.TIDEDesignPackage.Load (Line 866, "PasCppPakMgr.pas" + 42) + $0
[21475D71]{delphicoreide120.bpl} PasCppPakMgr.TIDEDesignPackage.DelayLoad (Line 748, "PasCppPakMgr.pas" + 11) + $4
[2147D48C]{delphicoreide120.bpl} PakList.TPackageListItem.LoadWait (Line 849, "PakList.pas" + 3) + $4
[2147D410]{delphicoreide120.bpl} PakList.TPackageListItem.LoadDesignPackage (Line 827, "PakList.pas" + 15) + $5
[2147C623]{delphicoreide120.bpl} PakList.TPackageListItem.SetIsInstalled (Line 531, "PakList.pas" + 7) + $3
[2147C3F9]{delphicoreide120.bpl} PakList.TPackageList.AddPackage (Line 445, "PakList.pas" + 13) + $5
[214C5FAE]{delphicoreide120.bpl} BasePasProjOpts.TProjectOptions.IsAPackage (Line 3006, "BasePasProjOpts.pas" + 1) + $B
[214C3EBB]{delphicoreide120.bpl} BasePasProjOpts.TProjOptsManager.InstallPackage (Line 2053, "BasePasProjOpts.pas" + 8) + $5
[214C5F97]{delphicoreide120.bpl} BasePasProjOpts.TProjectOptions.InstallPackage (Line 3001, "BasePasProjOpts.pas" + 0) + $3
[214E5165]{delphicoreide120.bpl} PasMgr.TPascalPackageCodeUpdater.InstallPackage (Line 11369, "PasMgr.pas" + 17) + $19
[2146CEED]{delphicoreide120.bpl} PkgContainers.TStdPackageProjectContainer.CommandHandler (Line 153, "PkgContainers.pas" + 5) + $5

Two methods, TPascalPackageCodeUpdater.InstallPackage and TProjectOptions.InstallPackage, both in delphicoreide120.bpl, looked very promising. I started Dependency Walker and loaded delphicoreide120.bpl from the Delphi bin directory to find if I could use one of them.

In Dependency Walker, I could locate the two functions as exports:

@Pasmgr@TPascalPackageCodeUpdater@InstallPackage$qqrv
@Basepasprojopts@TProjectOptions@InstallPackage$qqrx20System@UnicodeString

It looks as if these methods are defined as:

procedure TPascalPackageCodeUpdater.InstallPackage;
procedure TProjectOptions.InstallPackage(PackageName: string);

But both routines are methods, and a method has one extra, hidden parameter, the Self pointer, which is a reference to the instance of the object. The method needs this to be able to access the members of the instance. But how can I get at the instance?

The name TProjectOptions sounded a lot like IOTAProjectOptions, and fortunately, I could locate the following exports in delphicoreide120.bpl as well (plus a few others, but their names are far too long to properly display them here):

@Basepasprojopts@TProjectOptions@IOTAProjectOptionsConfigurations_GetActiveConfiguration$qqrv
@Basepasprojopts@TProjectOptions@IOTAProjectOptionsConfigurations_GetBaseConfiguration$qqrv
@Basepasprojopts@TProjectOptions@IOTAProjectOptionsConfigurations_GetConfiguration$qqri

So I gathered that I could try to get at the base address of the instance by obtaining a IOTAProjectOptionsConfigurations and then trying to get the implementing object of that interface. If you look at the diagram (from my article on pointers) below, you’ll see that an interface is a pointer (MyEditable) to a pointer (hidden IEditable field of the instance) to an array of pointers to pieces of code, i.e. the stubs that change the offset and then jump to the method code directly.

I experimented a bit and found that the stubs generated by the compiler looked like:

    add     eax,$FFFFFFE0              // 05 E0 FF FF FF
    jmp     TImplementingObject.Method // E9 xx xx xx xx

Eax is the Self pointer passed to the interface method, but the interface pointer points into the implementing object (in the diagram, to address 50016) and not to its base (50000, in the diagram). That is why the stub adds -16 to eax to make it a pointer to the base of the instance, since that is the Self pointer methods of objects need.

If I can get at the stub, I can read the offset used in that stub code (in the case above, $FFFFFFE0 or, in decimal, -16) and then adjust the interface pointer to make it a pointer to the base object. In other words, I treat the stub code as a piece of data and read the offset the stub uses directly from the code:

type
  TStub = packed record
    Add: Byte;       // Must be $05 - opcode for add
    Offset: Integer; // Offset I need
  end;
  PStub = ^TStub;

  TInterface = packed record
    QueryInterface: Pointer;   // IInterface methods are a little different.
    AddRef: Pointer;
    Release: Pointer;
    Method: PStub;             // Points to stub for first user method.
  end;
  PInterface = ^TInterface;
  PPInterface = ^PInterface;

TInterface is the array of pointers to stubs. The first three methods are always present, and are stdcall calling convention, so a little more complicated and different. But the first user method comes directly after these three, and that points to a stub as described above. TStub is the first part of the code, and a PStub is a pointer to it. Now I could obtain a IOTAProjectOptionsConfigurations from the IOTAProjectOptions I already have, and get the offset for the TProjectOptions instance:

var
  Stub: PStub;
  Configs: IOTAProjectOptionsConfigurations;
  ...
begin
  Stub := PPInterface(Configs)^^.Method; // double indirection, see diagram
  if Stub^.Add = $05 then
    Offset := Stub^.Offset
  else
  begin
    MessageDlg(SProjectOptionsNotFound, mtError, [mbOK], 0);
    Exit;
  end;

Well, that looked good, and worked fine in a test program, but it didn’t work here. Stub^.Add was not $05. So I displayed the content of the stub with ShowMessage and found that Add was $83 and Offset was $00E9C0C0. IOW, the bytes were 83 C0 C0 E9 00. Disassembling this led me to a different stub:

    add     eax,$C0  // 83 C0 C0 -- operand is a shortint
    jmp     xxxxxxxx // E9 xx xx xx xx

The delphicoreide120.bpl package apparently used a different stub, but with exactly the same effect. The opcode was 2 byte now (83 C0), and the offset a Shortint (-128..127). I had to revise my code a little:

  TStub = packed record
    case Byte of
      0: (LongAdd: Byte;       // Must be $05
          LongOffset: Integer);
      1: (ShortAdd: Word;      // Must be $C083
          ShortOffset: Shortint);
  end;
  PStub = ^TStub;
  ...
var
  Stub: PStub;
  Configs: IOTAProjectOptionsConfigurations;
  ...
begin
  ...
  Stub := PPInterface(Configs)^^.Method;
  if Stub^.LongAdd = $05 then
    Offset := Stub^.LongOffset
  else if Stub^.ShortAdd = $C083 then
    Offset := Stub^.ShortOffset
  else
  begin
    MessageDlg(SProjectOptionsNotFound, mtError, [mbOK], 0);
    Exit;
  end;

Now all was clear. I could finally load and call the function in delphicodeide120.bpl:

type
  // Method declared as normal function, so Self is explicit and can be changed.
  TPackageInstaller = procedure(Self: Integer; FileName: string);
  ...
var
  Installer: TPackageInstaller;
  ...
begin
  ...
  // Build package and install it
  Builder := InstallProject.GetProjectBuilder;
  if Builder.BuildProject(cmOTAMake, False, True) then
  begin
    // Find TProjectOptions.InstallPackage method in delphicoreidexxx.bpl.
    Module := LoadPackage(
      (BorlandIDEServices as IOTAServices).GetBinDirectory +
      PathDelim + 'delphicoreide120.bpl');
    if Module <> 0 then
    begin
      Installer := GetProcAddress(Module,
        '@Basepasprojopts@TProjectOptions@InstallPackage$qqrx20System@UnicodeString');

      // Call TProjectOptions.InstallPackage(BplFullName)
      // Address in interface + offset = address of TProjectOptions instance,
      // to be passed as Self instance pointer for method.
      Installer(Integer(Options) + Offset, BplFullName);
      UnloadPackage(Module);
    end
    else
    begin
      MessageDlg(Format(SDelphiCoreNotFound, ['delphicoreide120.bpl']),
        mtError, [mbOK], 0);
      Exit;
    end;

That finally installed the test components I had created and the package showed up in the package list!

Now all I had to do was pretty it up a bit, turn the names of the function and the IDE package into string constants, and provide some diagnostics showing which components were installed, since the IDE would not always do that.

Delphi 2010

Except for the suffix (140, instead of the expected 130) to the name of the delphicoreide package, nothing big had to be changed.

Delphi 2007

Delphi 2007 does not have a IOTAProjectOptionsConfigurations type at all, so I decided to try to use the original IOTAProjectOptions I already had, and fortunately, that works for both versions of Delphi. Of course I also had to change the name of the IDE package to delphicoreide100.bpl and the name of the function to @InstallPackage$qqrx17System@AnsiString. After these changes, nicely tucked up in a block of conditional compiler expressions, and a few minor, mostly cosmetic changes, it works equally well in Delphi 2007 and 2009.

Delphi 2006

In Delphi 2006, the Delphi 2007 version of the wizard worked remarkably well. After all, both have the same compiler and RTL. Only one function, TaskMessageDlg, that did not exist in Delphi 2006, had to be defined, nicely wrapped in a conditional directive. Otherwise, there were no problems.

Rudy Velthuis