FileInfo is a stateful reference type, so overriding Equals and GetHashCode to use value equality would be a Bad Idea™.
E.g.:
var f = new FileInfo("image.jpg");
var set = new HashSet<FileInfo> { f };
f.MoveTo("image2.jpg");
Console.WriteLine(set.Contains(f)); // Would be false (sometimes)
E.g.: