Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow to serve http files containing variable(s) #9485

Open
azr opened this issue Jun 26, 2020 · 9 comments
Open

allow to serve http files containing variable(s) #9485

azr opened this issue Jun 26, 2020 · 9 comments

Comments

@azr
Copy link
Member

@azr azr commented Jun 26, 2020

This has precedent: #3961 #1673 #2867 #4530 but was never exactly solved.

I would be nice if the contents served in HTTP by packer for the boot_command could be passed from a variable.


Why ?

Very often in packer buildfiles I see the boot command using local files, for example:

http_directory = "preseed_folder/"
boot_command   = [
    // ...
    " preseed/url=http://{{ .HTTPIP }}:{{ .HTTPPort }}/preseed.cfg<wait>",
    // ...
]

Very often a preseed.cfg file for packer will contain the following settings:

d-i passwd/user-fullname string vagrant
d-i passwd/user-uid string 1000
d-i passwd/user-password password vagrant
d-i passwd/user-password-again password vagrant
d-i passwd/username string vagrant

This results in a lot of images actually having vagrant/vagrant as login. This is okay in most situations as a provisioning step will change this setting later on but making these actual variables could make the process a bit safer by may be passing ssh keys and faster by setting these credentials correctly initially.

@azr
Copy link
Member Author

@azr azr commented Jun 26, 2020

A proposal for this would be to add a new http_files setting that can be used like so:

http_files = {
  "some_file" = "contents"
  "preseed.cfg" = <<EOF
d-i passwd/user-fullname string ${var.user_name}
d-i passwd/user-uid string 1000
d-i passwd/user-password password ${var.user_password}
d-i passwd/user-password-again password ${var.user_password}
d-i passwd/username string ${var.user_name}
EOF
# this could also be an ssh public key
}

This would start an http server that simply serves these string.
This would work in JSON or HCL2.

I'd keep it simple by not adding support for subfolders which seems like a 'nice-to-have' but I think it would be unnecessary.


For implementation:

I'd change these structs to have that new HTTPFiles map[string]string field:

// Packer will create an http server serving `http_directory` when it is set, a
// random free port will be selected and the architecture of the directory
// referenced will be available in your builder.
//
// Example usage from a builder:
//
// `wget http://{{ .HTTPIP }}:{{ .HTTPPort }}/foo/bar/preseed.cfg`
type HTTPConfig struct {
// Path to a directory to serve using an HTTP server. The files in this
// directory will be available over HTTP that will be requestable from the
// virtual machine. This is useful for hosting kickstart files and so on.
// By default this is an empty string, which means no HTTP server will be
// started. The address and port of the HTTP server will be available as
// variables in `boot_command`. This is covered in more detail below.
HTTPDir string `mapstructure:"http_directory"`
// These are the minimum and maximum port to use for the HTTP server
// started to serve the `http_directory`. Because Packer often runs in
// parallel, Packer will choose a randomly available port in this range to
// run the HTTP server. If you want to force the HTTP server to be on one
// port, make this minimum and maximum port the same. By default the values
// are `8000` and `9000`, respectively.
HTTPPortMin int `mapstructure:"http_port_min"`
HTTPPortMax int `mapstructure:"http_port_max"`
// This is the bind address for the HTTP server. Defaults to 0.0.0.0 so that
// it will work with any network interface.
HTTPAddress string `mapstructure:"http_bind_address"`
}

type StepHTTPServer struct {
HTTPDir string
HTTPPortMin int
HTTPPortMax int
HTTPAddress string
l *net.Listener
}

One simple solution would be to serve them using a map[string]string:

func (m MyMap) handle(w http.ResponseWriter, r *http.Request) {
  file, found := m[r.URL.Path]
  // ....
}

Note that the map implementation makes the subfolder serving easy as we would just match the whole string.

Another solution ( which I like a bit less ) would be to create these files in a temporary folder and serve that folder.

@azr azr changed the title allow to serve boot command from variable allow to serve http files from variable Jun 26, 2020
@azr azr changed the title allow to serve http files from variable allow to serve http files containing variable(s) Jun 26, 2020
@jhawk28
Copy link
Contributor

@jhawk28 jhawk28 commented Jun 27, 2020

Would it be simpler to use the golang template package (https://golang.org/pkg/text/template/) and serve up files with a particular extension?

@azr
Copy link
Member Author

@azr azr commented Jun 29, 2020

@jhawk28 I'm not sure to understand, text/template could be used to interpolate variables with the http_files I described before but it cannot serve these files.

@jhawk28
Copy link
Contributor

@jhawk28 jhawk28 commented Jun 29, 2020

@azr here is how I think it could work:

  1. The user creates a preseed.cfg.pkrtemplate file
  2. The packer.json file references {{ .HTTPIP }}:{{ .HTTPPort }}/preseed.cfg
  3. The http server sees the request for preseed.cfg and sees that there is a .pkrtemplate file for it. It executes the template with the variables and serves up the result.
@azr
Copy link
Member Author

@azr azr commented Jun 30, 2020

Ah I see, thanks for you suggestion, I like my suggestion a little bit better because it is a bit less magic and therefore easier to document & comprehend and if you were in HCL2 mode it would be super easy to put the template definition in a file on it's own, for example:

# preseed.cfg.pkr.hcl
locals {
  "preseed.cfg"  = <<EOF
d-i passwd/user-fullname string ${var.user_name}
d-i passwd/user-uid string 1000
d-i passwd/user-password password ${var.user_password}
d-i passwd/user-password-again password ${var.user_password}
d-i passwd/username string ${var.user_name}
EOF
  }
@jhawk28
Copy link
Contributor

@jhawk28 jhawk28 commented Jun 30, 2020

to each his own. I prefer the ability to encapsulate the file outside of the packer json/hcl file.

@SwampDragons
Copy link
Member

@SwampDragons SwampDragons commented Jun 30, 2020

Why not save the templates off as their own files and have a map[string]string var named http_templates -- Packer could load the map keys, interpolate them as golang templates, and save them off as the map values inside the http_dir. Then it's neither "magically" loading files with a specific name nor forcing users to dump whole preseed files into variables.

@jhawk28
Copy link
Contributor

@jhawk28 jhawk28 commented Jun 30, 2020

I don't think you need to render the templates as files, you can just serve them up in the HTTP handler. We could just identify the templates in a list. If it allowed wildcards, it would reduce the config to allow the user to define something like *.pkrtemplate.

Example:

{
  "http_templates": ["preseed.cfg", "*.pkrtemplates"],
  "http_dir": "http"
}
@azr
Copy link
Member Author

@azr azr commented Jul 1, 2020

In HCL2 we could take the templatefile function from Terraform; this would allow to load template files in other places too, making the solution less specific and more general:

#preseed.cfg.tpl
d-i passwd/user-fullname string ${user_name}
d-i passwd/user-uid string 1000
d-i passwd/user-password password ${user_password}
d-i passwd/user-password-again password ${user_password}
d-i passwd/username string ${user_name}
# source.pkr.hcl
source "..." "..." {
  http_files = {
      "preseed.cfg" = templatefile("preseed.cfg.tpl", { user_name = var.user_name , user_password = var.user_password ),
  }
}

etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.