In general, if you have a small molecule document and you want to have access to the proteomics menu item, you should switch to "Mixed" mode instead of "Proteomics" mode.
However, that will not help you in terms of adding decoy peptides.
I am not sure that I understand why you had to increase the Method Match Tolerance M/Z to 0.5. That number controls how close the isolation window of an MS2 scan has to be to the calculated m/z of the precursor in order for Skyline to decide that that MS2 scan belongs to that precursor.
If you have to set that to a big number like 0.5, that means that the thing that generated your method had a very different idea of what these molecules were than what Skyline thinks.
Do you have any idea why that would be?
It might be helpful if you send us your Skyline document and at least one of your .raw files.
In Skyline you can use the menu item:
File > Share
to create a .zip file containing your Skyline document and supporting files including extracted chromatograms.
If that .zip file is less than 50MB you can attach it to this support request.
Otherwise, you can upload it here:
https://skyline.ms/files.url
Also, send us at least one of your .raw files.
-- Nick