I have found that most of the time, the reccomended fix is to add (on error resume next) before setting the shape, as I get an error saying it does not exist.

NO!

The recommended way to handle runtime errors is not to shove them under the carpet and continue execution as if nothing happened - which is exactly what On Error Resume Next does.

The simplest way to avoid runtime errors is to check for error conditions, and avoid executing code that results in 100% failure rate, like trying to run a method on an object reference that's Nothing:

For i = 1 To (a certain number)
    Set shp = f_overview.Shapes("btn_" & i)
    If Not shp Is Nothing Then shp.Delete
Next

In cases where you can't check for error conditions and must handle errors, the recommended way is to handle them:

Private Sub DoSomething()
    On Error GoTo CleanFail

    '...code...

CleanExit:
    'cleanup code here
    Exit Sub

CleanFail:
    If Err.Number = 9 Then 'subscript out of range
        Err.Clear
        Resume Next
    Else
        MsgBox Err.Description
        Resume CleanExit
    End If
End Sub

There is nothing WRONG in using OERN (On Error Resume Next) provided you understand what you are doing and how it is going to affect your code.

In your case it is perfectly normal to use OERN

Dim shp As Shape

For i = 1 To (a certain number)
    On Error Resume Next
    Set shp = f_overview.Shapes("btn_" & i)
    shp.Delete
    On Error GoTo 0
Next

At the same time ensure that you don't do something like

On Error Resume Next
<Your Entire Procedure>
On Error GoTo 0

This will suppress ALL errors. Use proper error handling as shown by Matt

Edit:

Here is another beautiful example on how to use OERN This function checks if a particular worksheet exists or not.

Function DoesWSExist(wsName As String) As Boolean
    Dim ws As Worksheet

    On Error Resume Next
    Set ws = ThisWorkbook.Sheets(wsName)
    On Error GoTo 0

    If Not ws Is Nothing Then DoesWSExist = True
End Function

If you wish you can also loop through all the sheets to check is the sheet exists or not!

Instead of trying to blindly delete shapes and skipping errors, why not run through the list of known shapes and delete them. Then you don't have to worry about an On Error Resume Next which often ends up being abused.

Sub Test(TheSheet As Worksheet)

Dim Shp as Shape

For Each Shp in TheSheet.Shapes
  If left(Shp.Name, 4) = "btn_" Then
    Shp.Delete
  End if
Next

End Sub

If you want to delete all shapes, remove the If statement. If you want to delete a number of differently named shapes, modify the If statement appropriately.

Try

On Error Resume Next

for i = 1 to (a certain number)
    Set shp = f_overview.Shapes("btn_" & i)
    if err<>0 then err.clear else shp.delete
next

on Error Goto 0

Sub test()
On Error Resume Next
Application.DisplayAlerts = False

For i = 1 To 5
    Set shp = f_overview.Shapes("btn_" & i)
    shp.Delete
Next
Application.DisplayAlerts = True
End Sub