Exporting a list of projects in Node.JS

function Project(project) {
  this.name = project.name;
  this.order = project.order;
  this.title = project.title;
  this.date = project.date;
  this.tags = project.tags;
  this.logo = project.logo;
  this.html = project.html;
  this.agency = project.agency;
  this.slides = Object.values(project.slides).map(slide => slide);
  this.path = `projects/${this.name}`;
  this.route = `/projects/${this.name}`;
  this.slidesPath = `/projects/${this.name}/slides/`;
  this.hiDefAffix = '@2x';
};

If all this does is just set props to an instance, then you probably don't even need a constructor. You can simply use a factory function that accepts a project object and returns an object with properties.

function createProject({name, order, title, date, tags, logo, html, agency, slides, name}){
  return {
    name,
    order,
    title,
    date,
    tags,
    logo,
    html,
    agency,
    slides: Object.values(project.slides),
    path: `projects/${this.name}`,
    route: `/projects/${this.name}`,
    slidesPath: `/projects/${this.name}/slides/`,
    hiDefAffix: '@2x'
  };
}

Object.values(project.slides).map(slide => slide);

Looks like project.slides is an object. Object.values creates an array of values. array.map just creates a copy of that array, which is useless.

const all = {};
Object.values(projectsData).map(project => all[project.name] = new Project(project));

You are misusing array.map. array.map is for transforming one array into another in a 1-to-1 transformation. What you are doing here is creating an object. This is a job for array.reduce.

const all = Object.values(projectsData).reduce((all, project) => {
  return { ...all, [project.name]: createProject(project)};
}, {});

try {
  if (this.all[project]){
    return this.all[project];
  } else {
    throw new Error(`Project "${project}" not found.`);
  }
} catch (error) {
  return false;
}

The try-catch is pointless. You are immediately catching the error and simply returning false. Why not return false in the first place?

return this.all[project] || false;

toJSON() {
  return JSON.stringify(this.all);
}

I believe this is a practice commonly done by Java developers to have an intermediate function to handle special properties. But this is unnecessary in most cases since any simple JS object is immediately serializable by JSON.stringify, and you don't appear to use any of the new data types.

How can I make this Javascript code more OO

This is the wrong way to think in terms of programming, locking yourself to a paradigm. What you should be asking is "How to make this code simpler?" or "How to make this code easier to understand?".

Also, OO isn't about classes, or constructors or methods and properties. It's about working with objects, whether they are instances of classes or simple data structures like object literals or arrays. You can do a lot with just object literals and arrays. You aren't going to need the class and methods fluff in most cases.