Extracting grid cells of a netcdf file based values from another nc file with R

I want to extract the grid cells from a MERRA2 netcdf file which are flagged as sea only with R:

However parts of the land areas are within my selection due to the curved nature of the lower boundary of the land area. I want to select the area of the sea area which are closest to the land but there are areas of the land in my selection which I want to filter out.

I have another nc file which has the fractions of the land and sea areas from which I understand it is possible to extract the grid cells I need from:

So I have to define a mask that I can use to only select ocean points from my first file with a certain threshold since there are fractional coverage of land and ocean. Please is there a way to do this with R or maybe in ArcGIS

you can do something using CDO perhaps. If you decide on a land fraction threshold "c" you want to use for your mask from your landsea mask file landsea.nc (it sounds from your description that c needs to be a very small positive real number) then you can either define a mask which is zero for sea and one for land

cdo lec,c landsea.nc mask.nc 

This sets all points with values < c to 1, otherwise 0 (for land)

Or if you want to then set all land points to "missing" then you can

cdo setrtomiss,0.5,2 mask.nc mask_miss.nc

Now land points are set to "missing"

you can then scale the data file with the mask to either set the land points to zero

cdo mul merra2.nc mask.nc out1.nc 

or you can set them to missing :

cdo mul merra2.nc mask_miss.nc out2.nc 

Which you can then process.

Of course, here I am assuming that the landsea.nc mask file is on the same grid as your merra file. If they are not you will need to regrid the mask file first, which you can also do with CDO.