You should be able to access your network from any connected device 100% of the time, wired or wireless.
Wired is faster, more reliable, more secure, and passes more bandwidth. The hassle is it requires cables, though powerline network adapters do *usually* work [it depends on your home's wiring], but they don't last. You plug one powerline adapter into an outlet near the router, plug in a network cable connecting it to the router, plug in another adapter to an outlet elsewhere in your home, mate the two, and you have one or more connectors to plug in network cables for whatever devices. Where you have more network cables you want to plug in than available sockets to plug them into, or when you want to run one network cable instead of many, you use switches. They're cheap -- ~$10 for one with 5 ports, which would let you plug in 4 cables for devices, & one to connect to the router [or another switch]. You can also buy cheap USB wired network adapters that work with otherwise wireless only devices -- I use one that's built into a small USB hub for my Windows tablet for big downloads, like win10 version updates. While network cables can be found cheap on sale, there are times when it can work best to make your own. You can find bulk cable on sale from time to time, and the connectors & crimper are usually cheap money. Attaching the connectors however can be a PITA -- be forewarned. I bought a $5 network cable tester & it's proved its worth many times over.
Wireless is both simpler and more complicated IMHO. There are different ways that they can use usually two frequency bands that *may* provide more bandwidth if the device allows it. Our devices, like the kindles, use one frequency at a time. One band goes further, but has lower bandwidth. Some routers choose the band to use automatically -- I have ours configured as two separate WiFi networks so I can control which band they use. To get a better signal in our bedroom I use a WiFi Mesh Extender -- it uses the same WiFi network, so you don't have to switch which network the device uses when you walk into the bedroom. You can also get powerline adapters with WiFi mesh extenders built in, but I opted not to since as I said, they don't last. WiFi Mesh networks use 2 or more units that work cooperatively to provide more coverage, but they're more expensive, and depending on your perspective, may have privacy issues, often [Usually?] contacting the company that makes them.
To set up a new router as possible consider its location -- higher vs. lower, with a prospective WiFi signal path that won't be blocked ideally. There are a few WiFi signal strength apps for Android that are very handy, e.g. you can walk around your home and find weak spots. Moving the router a few feet can matter. [Note that you *may* also be able to swap out the WiFi antennae(s), which won't give you a more powerful signal, but can shape coverage, e.g. giving up vertical range for horizontal.] How you log into a router initially can vary by manufacturer -- some require special software, I've seen them require an Android app, but usually [hopefully] you just need to put the address in the address bar of a web browser. It can unfortunately be a challenge to find the default name & password to log in [I know it was for me with our current Netgear router]. One of the 1st things you want to do is change those. The rest of the router's settings vary from one make/model to the next, but it isn't hard to find answers with Google -- I suggest leaving your existing router connected, so you can still get online, plug the new one into the old, and connect one device to the new router to set it up. Then when the basic setup's done, pull the old one.
For WiFi, with the old router disconnected, there's no reason you can't use the same network name [SSID] & password. Do that and every device that connected before should connect automatically now. I set ours up for two networks as mentioned, & since most of our devices could use 5GHz, re-used the SSID & password for that band -- I had to set up the one or two devices that couldn't use 5GHz to the 2nd SSID. Your router & devices might also allow for a quick push button pairing, not unlike Bluetooth pairing that can make it easier than trying to enter a long password. Some routers allow you to control what devices connect by entering their MAC address, but this is just a gimmick when it comes to security with no real benefit.
Finishing up, you may find that some of your devices need to have a permanent IP address, rather than whatever the router assigns each time -- usually you can look at the IP addresses currently being used, figure out which one goes with which device, and set it to always use that IP address for that device's MAC address. I had to do that with our TV for example, to avoid having to set up the network whenever it connected. If you have neighbors with interfering WiFi networks that can hurt your WiFi performance. There are as mentioned Android apps that show you the amount of WiFi traffic per channel as you walk around your home -- you can usually manually select the WiFi channel your router uses to minimize interference.
