Is it possible to run a modal analysis in a way that results are calculated just for the part of the model you are interested in (ignoring modes of the rest of it)?
If there is a clear separation plane (i.e. across which modes do not develop), you could calculate the stiffness of the surrounding structure and apply spring-damper elements at the separation line after removing the structure that isn't of interest. The interface's bending stiffness is also important.
It makes the analysis more questionable though.
You can create a group in Femap, picking the elements for the portion of the model that you are interested in. Then use "add related" to bring in all nodes,materials,properties etc needed by those elements.
Now create a modal analysis and on the "bulk data form" at the top, instead of the default "full model" select your group of interest. The analysis now includes only that portion of the model.
Thanks for your comment fembraking.
Would it account for the stiffness of the boundary interface? If not, would it be possible to achieve it somehow without using superelements? I read something about asets but it is not very clear to me.
No, my suggestion would be the modes of your component with no influence from the surrounding structure. If you want to understand more about what an ASET is and does, then I would suggest reading about "static condensation" or "Guyan reduction", there are many sources including the NX Nastran help documents.
Depending on your intent with this analysis, you have a few options. One simple option is to constrain the nodes attaching to the adjacent structure. This would be the "fixed boundary modes" of your component.
Without a superelement license, another option, would be to set up an external superelement creation run for your "base structure" using the component attach locations as the ASET. Use the DMIGPCH option on the EXTSEOUT command, and the resulting punch file will contain the reduced stiffness at those locations. You can then use that as DMIG input to include this boundary stiffness( no SE license is required for K2GG input), or you could possibly use the stiffness terms to create CBUSH springs at the boundary.
A complete description of what you you are trying to accomplish would make better suggestions possible.