Shouldn't be a big deal, I guess no script uses the delete bookmark operator, and for 2.7x, breaking the api should be ok anyway.

There's:
bpy.ops.file.bookmark_add()
bpy.ops.file.bookmark_toggle()

but:
bpy.ops.file.delete_bookmark()

bookmark_* follows overall convention (matrix, matrix_basis, matrix_world, ...)

so make it bookmark_delete() !