Tagged Questions

1
vote
1answer
24 views

Backpropagation for Heart Disease Prediction - Results not accurate?

I'm using Multilayer Perceptron with backpropagation to predict heart diseases, and I am using the dataset linked here: https://drive.google.com/file/d/1LGSDCODkoj-bqhvdCLMeJu7BbTyzyjyE/view?usp=...
6
votes
1answer
172 views

Compute gradients for each time step of tf.while_loop

Given a TensorFlow tf.while_loop, how can I calculate the gradient of x_out with respect to all weights of the network for each time step? network_input = tf.placeholder(tf.float32, [None]) steps = ...
-5
votes
1answer
53 views

How do you train a neural network without an exact answer? [closed]

Most neural networks use backpropagation to learn, but from how I've understood it you need an exact answer to what the outputs should be for this to work. What I want to do is to learn a walker bot ...
0
votes
1answer
30 views

How to determine the size of bias matrices in a neural network?

I'm new to the world of machine learning. My question is how can I determine the size of the biases in a neural network (with backpropagation algorithm)? Currently, I have a 2-layer neural network (1 ...
0
votes
1answer
23 views

Tensorflow TypeError: Fetch argument None has invalid type <class 'NoneType'> while finding gradient

It's a text analysis task with CNN model,I want to visualize which word is triggering most to my particular classification,Here is my code with tf.Session() as sess: saver = tf.train....
1
vote
0answers
20 views

simple implementation of Adam in NN

Please look at this pseudo python code. It is just for ilustration. import numpy as np X = np.array([[1,0],[0,0]]) y = np.array([1,0]) w0 = np.random.random((2,1)) learning_rate= 0.01 for i in ...
2
votes
1answer
48 views

Create and name matrices inside a loop given a simple row vector

I am trying to implement backpropagation by my own. Suppose I have a simple vector with whole numbers like the following: x=[5,3,6,4] What I would like to do is create matrices, using the previous ...
1
vote
0answers
48 views

What makes a (too large) neural network diverge?

I've build a neural network (NN) having only 6 inputs and 8 classes (it is thus a classifier I'm building). To give some more background: The number of training samples is 80,000 samples as is the ...
-2
votes
0answers
19 views

how to make double backpropagation with thensorflow? there are always an error that “'NoneType' object has no attribute 'pred'”

# -*- coding: utf-8 -*- """ Created on Wed Jul 12 21:58:39 2017 @author: vivid3d """ import os import numpy as np import tensorflow as tf import input_data import model import math import tensorflow....
0
votes
1answer
69 views

Returning mutiple values in the input function for `tf.py_func`

I'm trying to set custom gradients using tf.py_func and tf.RegisterGradient. Specifically, I'm trying to take a gradient of an eigen value w.r.t its Laplacian. I got the basic thing working, where my ...
0
votes
1answer
42 views

What's wrong with my backpropagation?

I'm trying to code a neural network from scratch in python. To check whether everything works I wanted to overfit the network but the loss seems to explode at first and then comes back to the initial ...
0
votes
0answers
62 views

Sigmoid derivative implementing Backprop for a Neural Network

I'm implementing a simple neural network following the Andrew Ng's tutorial in Coursera. I performed gradient checking in order to check the correction of my calculations of the gradients in my ...
0
votes
0answers
25 views

Get gradients of some certain nodes in Tensorflow

Suppose three functions x=f(a;w1), y=g(x;w2) and z=h(y;w3). Tensorflow will compute gradients automatically, using graph and back-propagation. dz/dx=dz/dy*dy/dx, dz/dw1=dz/dx*dx/dw1 etc. I'd like to ...
0
votes
2answers
42 views

Backpropagation in python: cannot understand a line of code

I am trying to read the following code for back propagation in python probs = exp_scores /np.sum(exp_scores, axis=1, keepdims=True) #Backpropagation delta3 = probs delta3[range(num_examples), y] -= ...
1
vote
1answer
73 views

When neural network loss is descending but the accuracy is not increased?

I implement a batch-based back-propagation algorithm for a neural network with one hidden layer and sigmoid activation function. The output layer is one-hot Sigmoid layer. The net of first layer is z1....
1
vote
0answers
25 views

backpropagation for gender classification model using python

I have created a model for gender classification using NLTK in python. I am using a feed-forward mechanism. How to incorporate a back propagation method into this? import nltk from nltk.corpus import ...
0
votes
0answers
91 views

I am having trouble training a simple neural network

I am learning to make neural network and I have derived the following equations for backpropagation. Is something wrong with this because I can't seem to get the neural network training. I am coding ...
1
vote
0answers
56 views

Trying to implement backpropagation

I'm very new to machine learning and am trying to implement gradient descent with back propagation. I have derived the following code. I don't know why its not working... def cost(params,X,y,alpha,...
3
votes
2answers
138 views

Matrix dimensions not matching in back propagation

Here I'm attempting to implement a neural network with a single hidden layer to classify two training examples. This network utilizes the sigmoid activation function. The layers dimensions and ...
2
votes
1answer
61 views

Dimensions of 2 hidden layer neural network not correlating

Here I'm attempting to implement a 2 layer neural network using numpy alone. Code below is just computing forward propagation. The training data is two examples where the inputs are 5 dimensions and ...
2
votes
1answer
46 views

Linear Function in Neural Network is producing large value as output

I am performing regression on the iris data set to predict its type. I have successfully performed classification using the same data and same neural network. For classification, I have used tanh as ...
0
votes
1answer
33 views

Backpropagating bias in a neural network

Following Andrew Traks's example, I want to implement a 3 layer neural network - 1 input, 1 hidden, 1 output - with a simple dropout, for binary classification. If I include bias terms b1 and b2, ...
2
votes
1answer
204 views

Dropout: why is my neural network not working?

I'm using normalized MNIST dataset (input features = 784). My network architecture is 784-256-256-10: two hidden layers of 256 neurons each using sigmoid activation functions, and softmax activation ...
3
votes
1answer
41 views

When will the computation graph be freed if I only do forward for some samples?

I have a use case where I do forward for each sample in a batch and only accumulate loss for some of the samples based on some condition on the model output of the sample. Here is an illustrating code,...
0
votes
2answers
103 views

How to find and understand the autograd source code in PyTorch

I have a good understanding of the autograd algorithm, and I think I should learn about the source code in PyTorch. However, when I see the project on GitHub, I am confused by the structure, cuz so ...
2
votes
0answers
60 views

Custom gradients for vector binarization in tensorflow

I am trying to implement the "feed-forward convolutional/deconvolutional residual encoder" as described in this paper: https://arxiv.org/abs/1511.06085 In the network architecture they use a ...
0
votes
0answers
48 views

Minimize cost from gradient descent algorithm

I'm working on an optimization problem. My function is like this: C=-σ(wx+b)+σ(wθz+b) where σ is a sigmoid function. What I'm doing here is to find an optimal value of w and b so that my cost C can ...
1
vote
1answer
44 views

Neural net termination rule

I want to build my first neural net for recognazing hand-written digits in python, but I cant find a good, simple termination rule? What I mean by "termination rule" is when to stop updating my ...
0
votes
1answer
97 views

Autoencoder using Backpropagation

I'm trying to implement autoencoder using this resource which implements backpropagation algorithm. I'm using the same feed forward algorithm implemented there but however it gives me a large error. ...
3
votes
1answer
198 views

Backpropagation with Momentum

I'm following this tutorial for implementing Backpropagation algorithm. However, I am stuck at implementing momentum for this algorithm. Without Momentum, this is the code for weight update method: ...
2
votes
1answer
116 views

Understanding Michael Nielsen's backpropagation code

I'm trying to understand/run the code in Michael Neilsen's Neural Networks and Deep Learning chapter 2, on backpropagation: http://neuralnetworksanddeeplearning.com/chap2.html#...
0
votes
1answer
73 views

Can somebody help me to correctly derive the loss function?

I'm trying to adapt the example from http://cs231n.github.io/neural-networks-case-study/#together to make a neural network for a numeric target variable so it will be a neural network with regression. ...
1
vote
0answers
76 views

Backpropagation in Python 3.6

I tried to implement a neural network that uses backpropagation for training it. As the title says I did it in Python 3.6, and tried to make it learn the XOR function. When I run the code and test ...
1
vote
1answer
49 views

Addition of hidden layer makes output converge on one value — Tensorflow

I started with a simple linear regression-style net, written in Tensorflow and largely based on their MNIST for beginners tutorial. There are 7 input variables and 1 output variable, all on a ...
2
votes
2answers
69 views

Neural Network seems to be getting stuck on a single output with each execution

I've created a neural network to estimate the sin(x) function for an input x. The network has 21 output neurons (representing numbers -1.0, -0.9, ..., 0.9, 1.0) with numpy that does not learn, as I ...
2
votes
2answers
52 views

Neural Network with two neurons

I was trying to implement a simple neural network from scratch using python. This neural network has only two neurons and the task is to match the input to output. (i.e. x = 0 --> output = 0, x = 1 -->...
0
votes
0answers
117 views

How to Backpropagate at Fully Connected Layer in CNN

I am trying to build a pet project based on Convolutional Neural Network in python which basically tells if the given image is X or O. My architecture is as follows: [INPUT - CONV - RELU - POOL - FC - ...
2
votes
1answer
50 views

Backpropagation: Why doesn't error approach zero when it is multiplied by deriv of sigmoid?

I'm trying to implement backpropagation to my simple neural network which looks like this: 2 inputs, 2 hidden(sigmoid), 1 output(sigmoid) . But it doesn't seem to work properly. import numpy as np #...
2
votes
0answers
37 views

Torch's loss.backward() hanged on ParlAI

I am interested in Memory Networks and Movie Dialog QA. Recently facebook announced AI training framework called ParlAI, which supports many models and datasets. Although I tried below command on ...
1
vote
0answers
189 views

input & target tensors for recurrent neural network in tensorflow and optimal TBPTT

I have a few fundamental questions about RNNs in tensorflow (v1.3 at time of writing). I have a pretty solid understanding of RNNs (+LSTM) I think. Amongst other things I've read Sutskever's PhD ...
2
votes
1answer
431 views

MLPClassifier parameter setting

I'm developing a project which uses Backpropation algorithm. So I'm learning Backpropagation algorithm in scikit-learn. mlp = MLPClassifier(hidden_layer_sizes=(hiddenLayerSize,), solver='lbfgs', ...
2
votes
0answers
150 views

Neural Network in python only learning for 1 in-/output pair per dataset?

I wrote code for generating and training artifiacial neural networks in python recently (with backpropagation). It all works very well, but this one problem bugs me for days now and I can't seem to ...
3
votes
0answers
434 views

Saliency Map for Keras CNN LSTM (TF Backend)

I am trying to get the saliency maps from my neural network, but I am struggling a little. My network does DNA Classification (similar to text classification), and is sequential as follows: Conv1D-> ...
1
vote
1answer
68 views

I can't find the bug in this implementation of backpropogation?

My data is 4123 rows of inputs and outputs to an xor gate. I want to write a Neural Network with three input layer neurons (the third one is bias), a hidden layer, and an output layer. Here's my ...
1
vote
1answer
92 views

Basic Backpropagation Implementation Not Working

I'm in the early stages of understanding backpropagation and I attempted to implement it myself. The dataset I attempted to work with was the iris dataset of size (150, 4). I'm only worried about ...
1
vote
0answers
34 views

why not delete 0.0 here?

downstream_delta = reduce( lambda ret, conn: ret + conn.downstream_node.delta * conn.weight, self.downstream, 0.0) I wanna know why there exists 0.0. The self.downstream here is a ...
0
votes
0answers
50 views

Backpropagation in Python

I want to write a neural network using Python. I'm using FNN and simple test values. But it shows wrong result on the sample set. What is the error? net = Net(1, 1, 1) # number of input, hidden and ...
2
votes
2answers
73 views

how can I do a back propagation for a simple neural net with sigmoid activation?

I am a beginner in deep learning. I am currently struggled with the back propagation algorithms. I found this piece of code online of the back propagation of a simple neural net with sigmoid ...
0
votes
1answer
120 views

RNN Training Loss Goes Down Then Acts Erratically

I've been trying to create a vanilla 3 layer RNN in Python 3.6, but whenever I train it the training loss decreases for the first X amount of training iterations then goes up and down sporadically and ...
0
votes
0answers
46 views

Issues with backpropagating XOR neural network in Python

I have the following python code which implements a simple neural network (two inputs, one hidden layer with 2 neurons, and one output) with a sigmoid activation function to learn a XOR gate. The code ...

1 2 3