So you were the minority then? By the sounds of it the Caucasian population wasn't big enough to be a threat.
In the small Nebraska town that I lived in there was definitely an "other side of the tracks". Mexicans lived on one side, whites on the other. At one point in time I lived on the border. While in 6th grade, I'd be outside playing in my yard and a group of Mexicans walk by heading to their side of the town. I got the proverbial "what you looking at puta" claims and threats of getting beat down simply for being in my front yard. This happened a few more times before my single mother decided it was best to move further away from the Mexican part of town. On the flipside, I dated a half-Mexican girl whose father was a short-haul trucker and worked with my grandfather. One night over dinner my grandfather said "you know she's a wetback, right? Her father is Mexican." After picking up my jaw off the floor I acknowledged him and transitioned the conversation to a different topic. The co-worker of my grandfather was a "friend" but from the tone of his voice you could tell this old German man still felt the Mexicans were lower status in some way.
In Laredo the other side of the tracks (from my experience) was just school rivalry stuff. I do remember being bullied by a white kid and a Hispanic kid in the 8th grade at my middle school working together to push me and my friend (a white kid around). But my take was they were football players and we weren't. We were in the band. Race had nothing to do with it. I also recall being involved in a scrap in Rio Grande City in the end zone during a football game! One of our guys (a Hispanic guy) came over to us in the stands (I wasn't in the band anymore) and said so and so's girl friend had been messed with (whatever it was) and he picked out a few of us to go with him to deal with it. I recall being terrified but compliant. We walked over to the concession stand and they saw us coming and it appeared half the stands emptied to greet us in the end zone. We had about seven or eight of us and I'd say we were 50/50 white/Hispanic. One of their guys walked up and smacked one of our guys in the face with a clean roundhouse. I was standing right next to him. My friend stood his ground, a lot of glaring and cussing ensued and we quickly retreated back to our side of the field.
In the end, I had friends. Their race didn't matter.