Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not keep the build.zig cache manifest file locked. #7041

Merged
merged 1 commit into from
Dec 10, 2020

Conversation

FireFox317
Copy link
Contributor

@FireFox317 FireFox317 commented Nov 9, 2020

This allows to have multiple instances of zig build at the same
time. For example when you have a long running zig build run and
then want to run zig build somethingelse.

I tried to implement this on windows, however when having a long running command (zig build run) and then running the next command (zig build another) immediately exits with code 1 without printing anything. This also happens before I applied this patch, so it is a different issue. However I don't have a debug build of zig on windows to debug this issue.

Note on Windows we need some other solution, because we can't change a running executable (build runner):

18:36 <andrewrk> hmm I think there is a new consideration though - with the updated caching behavior, this might actually cause a problem on windows where the exe can't be touched because it is being executed. previously we did not have this situation because hashing the input files was part of the cache namespace
18:37 <andrewrk> just brainstorming here, but one possibility would be to copy the `build` executable (which is likely to be small) to a unique file name before executing it, and then release the lock
18:38 <andrewrk> this would essentially match the semantics of the old cache behavior but with the performance benefits of the new one
18:38 <andrewrk> (in this proposal the unique file name would be based on the hashes of the input files)

Related: #7042

@FireFox317
Copy link
Contributor Author

Turns out #3825 caught me again... That is why on Windows it just exists with status code 1, without printing anything.

This allows to have multiple instances of `zig build` at the same
time. For example when you have a long running `zig build run` and
then want to run `zig build somethingelse`.
src/main.zig Outdated
Comment on lines 2422 to 2443
// To be able to have multiple instances of `zig build` running on Windows with a difference in
// the source of the `build.zig` file, we have to copy the executable to a name that includes
// a hash of the content of the `build.zig` file and then execute that file, instead of the
// original build executable.
const build_zig_file = try build_directory.handle.openFile(build_zig_basename, .{});
defer build_zig_file.close();

var bin_digest: [Cache.bin_digest_len]u8 = undefined;
try Cache.hashFile(build_zig_file, &bin_digest);

var file_name: [Cache.hex_digest_len]u8 = undefined;
_ = std.fmt.bufPrint(&file_name, "{x}", .{bin_digest}) catch unreachable;

const temp_file_basename = try std.zig.binNameAlloc(arena, .{
.root_name = &file_name,
.target = target_info.target,
.output_mode = .Exe,
});

const dir = comp.bin_file.options.emit.?.directory.handle;
_ = try fs.Dir.updateFile(dir, exe_basename, dir, temp_file_basename, .{});

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we only do this on Windows or also on other platforms?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. I think we should try to keep the behavior the same across platforms, smoothing over this particular difference.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can also check before compiling if the build executable is able to be written to (thus not being executed) and only then overwrite the original build executable and otherwise do the method I coded in. In that way there is only a bit over overhead in storage when running multiple zig build commands at the same time and most of the time you are running only one command at the same time anyway, so then you can just overwrite the build executable anyway.

@andrewrk
Copy link
Member

andrewrk commented Dec 9, 2020

Turns out #3825 caught me again... That is why on Windows it just exists with status code 1, without printing anything.

Well at least it's fixed now! You can update the dev kit by adding --prefix path/to/dev/kit to the zig build line.

src/main.zig Outdated
});

const dir = comp.bin_file.options.emit.?.directory.handle;
_ = try fs.Dir.updateFile(dir, exe_basename, dir, temp_file_basename, .{});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since you're not updating anything:

Suggested change
_ = try fs.Dir.updateFile(dir, exe_basename, dir, temp_file_basename, .{});
try dir.copyFile(exe_basename, dir, temp_file_basename, .{});

@andrewrk andrewrk merged commit 4c51ade into ziglang:master Dec 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants