I know how you feel, Bella. Our kids rooms are right next door and I can assure you that they kids sleep more soundly than we think.
In 7 months, my son's screaming has only woken my daughter once. And she's never woken him up.
The trick is consistency. You have to pick a plan and stick to it. If you quit because it's easier to just let her sleep with you, she'll know you're not serious.
As for her being scared, we give our daughter a glowstick. She hangs onto it and sleeps with it.
But I completely get that you don't want to deal with it at 2am. Our kids sound like similar sleepers --- crappy ones haha Definitely be consistent and firm. No means no.