6/29 Camera Problems

[Claude Wilkerson]

i also was having problems with chrome and could not fix. a switch to firefox solved all problems

[John Hamman]

I cannot access cam on either Chrome or Firefox this morning - it was there for awhile. Getting error message - something about "cross domain access denied". They had just begun moving some cars over in the swamp when I lost it.
John

[Ed Horan]

Same here. Cameras in Chama are out. I'm using Chrome. Refreshing the page doesn't help.

[Bruce McDowell]

Yeah, someone kicked something. Or it's just Thursday in Chama.

[Sharon Evans]

Well, guess no departure video for today.

[Craig Kumler]

No cams working on safari!

[Jason Rose]

The network is down. Either a power failure or the connection is down.

I'm checking on it.

[Sharon Evans]

Thank you Jason for your efforts on the cams.

I will check before 215 gets in to see if its back and we can get the arrival video. Glad I was able to do some stills this morning before it went out.

[Jason Rose]

Service has been restored.

[Sharon Evans]

Good news Jason...

[Bruce McDowell]

Gonna be one of those days, Jason. Just dies again with the same "crossdomain" error.

[Tim Tennant]

soup-nazi-left.jpg (17.06 KiB) Viewed 11107 times

NO CAMERA FOR YOU TODAY! Just a little humor folks. There appears to be a bandwidth issue that is causing the railroad's telephones at the Chama Depot to not work which correlates to times when there are high numbers of viewers on the cameras. Plus the Friends have a work session going on meaning more people utilizing the network. The railroad is working on the issue of increasing bandwidth so they can answer their phones, cameras are functional and Friends members who want WIFI have it. We apologize for the inconvenience. John Engs is also on site this week and has been talking to railroad personnel in the depot. That is what we know at this time.

[Sharon Evans]

Well with the cams down again, no video of this afternoon's arrival.

[Jason Rose]

There appears to be a bandwidth issue that is causing the railroad's telephones at the Chama Depot to not work which correlates to times when there are high numbers of viewers on the cameras.

As John and I discussed, the amount of viewers has nothing to do with the bandwidth usage. The camera stream is hosted off site by a 3rd party. Something else is going on, either increased utilization, over-utilization or circuit degradation.

[Tim Tennant]

Well Jason something is going on. When the cameras are unplugged the depot's phones work and reservation agents can take calls and reservations. I obviously misspoke relative to bandwidth issue so that is my error but that is how it was explained to me. Thanks for the answer and clarification.

[Tim Tennant]

After just talking to John Engs there is indeed a bandwidth issue. Because the cameras take up a pretty good chunk of bandwidth when the reservation agents experience a high volume of calls coming in there isn't enough bandwidth and the phones stop ringing. That is why the cameras are unplugged. So the solution is to purchase more bandwidth which someone will have to bear the cost of. Latest clarification.

[Sharon Evans]

So, if I am understanding Tim correctly, what is going on with the cams resolves to the railroad needing their phones to make reservations, and to do so they are unplugging the cam feed. Hopefully they remember to plug them back in when the call volume is reduced.

[Jason Rose]

Long story short: there is a problem with insufficient bandwidth. This isn't caused by the cameras as no changes have been made since the Depot Cam was added in 2015. However, disabling the cameras alleviates the symptoms by freeing up bandwidth.

No one will argue that having the cameras in operation is more important than the railroads ability to conduct business. So, whatever the cause of the over-utilization, shutting down the cameras resolves the issue so the railroad can function.

There is a plan underway to increase the amount of bandwidth, but that will take some time. Perhaps months. As a compromise, I suggested shutting off the Depot Cam until the enhanced service is available. John Engs approved the idea and will present it to John Bush. The changes have been made and I'm waiting for approval from John Bush and the network to be reconnected.

If more bandwidth is needed, I'll shut down the north camera as well.

More later...

[Sharon Evans]

You know Jason, we did not seem to have this issue last year.

[Ed Horan]

South camera still down. Hope that's not the one being shut down.

[Sharon Evans]

Agree, would much rather have both tipple cams vs the depot. With the depot cam you have to fight that cooling fan. As far as capture, the depot cams is very limited as to what it gets while the south tipple cam gives you the full picture of what is happening from the house all the way into south yard.

[Bruce McDowell]

As a computer guy, it does make me wonder what changed in the last couple of days, never mind the months since the depot camera was added. The problem cropped up today when the load wasn't that high (40's on the tipple cams?) during morning evolutions. Didn't see the issue on opening day when the viewership was truly stunning - not even a pause all morning that day. Can't dispute the net effect that dropping a camera or camera's has on the phones and the rest of the network. But you just don't have such a dramatic change over a short period of time (literally since yesterday) without something happening.

If this were the East Broad Top I'd have one of my engineers run right over and take a peek...

[Jason Rose]

Ok... I goofed... sorry...

I'll get the Depot Cam turned down and the South Cam back on this evening.

[Ed Horan]

Thanks Jason.

[Jason Rose]

Ok. All set. Depot Cam is dark, the other two are hot.

We're playing this by ear for a few days. Railroads operations have to be the first priority.

[Bruce McDowell]

Well there we go - nice work as always Jason!

Fascinated to hear the ultimate resolution for this issue. When I get my team on the phone next I'll run the scenario by them and see if they have any suggestions. If you're so inclined and want to PM me any of the specifics of the configuration that could enhance our discussion. Can be a challenge to perform remote diagnosis, but - heck - that's what these guys do before stepping out the door and heading for the customer site.

[Jason Rose]

Will send you a PM, Bruce.

[Claude Wilkerson]

it's great to see cameras up. does the cameras bring in any extra business? just curious

[Tim Tennant]

Claude that is a great question. We believe it probably does and hope that is indeed the case. You also have people that watch the cams who will never probably come and ride the Cumbres & Toltec. It is great PR for the Friends and railroad. I believe a majority of the viewers have ridden the railroad. The key is that the Friends and the railroad see a benefit to having the cameras up. Thanks for asking.

[Mike Maguire]

The cameras certainly helped convince me (and, more importantly, my long-suffering partner) that a visit and ride would be a GOOD THING. Our latest trip to the US was planned so we could loop down to Chama from Durango, thus "doing" both remaining sections of the San Juan Extension. The C&T has a much more authentic "working railroad" feel to it than the D&S. I loved being able to wander freely around the yards and equipment, chatting with staff and volunteers and generally being a big kid again !

[Tim Tennant]

Mike, thank you for sharing your story. This is a good example of how the cameras were a positive factor in you actually coming to visit. I believe there are more stories like this out there. I also know there are those who will never come out to ride the C&TS as the cameras provide them with their daily C&TS fix. We will hope for the best that the web cameras continue to convince people to come ride the railroad. Have a great 4th of July!

[Rich Murray]

On the whole I believe that the cameras are an asset to both the RR and the Friends. There are those who for whatever reason can't get out to Chama but because of the cameras stay interested in the RR. And maintain their membership with the Friends.

[Ed Horan]

I'm not able to view the cameras today. Displayed is "Cannot load M3U8: crossdomain access denied". Another heavy reservation day maybe?

[Tom Casper]

working ok for me on Chrome at 7:34pm CDT

[Sharon Evans]

Was that this morning by chance? I had the stream freeze on me while recording the departure but I got no error message on Chrome. It worked all afternoon and I got the arrival without issue.

[Tom Casper]

i wonder if he has been trying the depot cam as that has been blocked.

Tom C.

[Claude Wilkerson]

i also had "Cannot load M3U8: crossdomain access denied" problems and worked all day to fix. just reset browser

[Ed Horan]

I just cleaned up my browser just now and the cross domain message is still there on the tipple cameras. Using Chrome.

[Sharon Evans]

Ed, the tipple cams are working fine on Chrome for me at this point.

[Claude Wilkerson]

how i solved my problem was to click on 3 dotes in top right corner then scroll down to advanced and scroll down to reset and click. this fixed my chrome problem but had to erase firefox and redownload.

[Ed Horan]

I guess I'm not a computer savy person. I've cleaned up everything and the cameras are working in my end. Thanks for the help!

[Ed Horan]

Well, the problem is back. Could view todays departure. Left the site, and when I came back the crossdomain access denied message appeared.

[Ed Horan]

Update,literally and figuratively. After a week of using Chrome's "incognito window", to watch the yard cameras, my computer got some updates last night that returns things to normal today. Yay!!

[Jason Rose]

Thanks Ed. I was beginning to suspect a bad browser update was playing a part in this debacle.

[Steve Forney]

I use Firefox and also got the "Cannot load M3U8: crossdomain access denied" message. Timestamp in lower corner of camera frame correct when loaded. Chrome worked OK. Hiway cams OK.

Updated Firefox, no fix. Upgraded Java, no fix. Updated Flash, no fix.

Went back to doing other things.

After seeing Ed's message this AM I checked. Error message gone, cams load and run OK.

I do not allow any automatic updates on any of my machines at any time. So no "automatic browser fix" here.

[Jason Rose]

Thanks for the info, Steve.