I think the easiest way would be to use Ftp.DeleteFiles method with RemoteSearchOptions parameter:
Ftp.DeleteFiles(RemoteSearchOptions.UseLambdaMatch(
(RemoteSearchItem item) => { return false; } )
You need to provide your own callback that would return false for 'my.config' name and true for everything else.