Why did Buck not want to go with Charles and Hal?

In relation to what story? Why not?