Tagged Questions
1
vote
0answers
21 views
Postgresql out of memory during transaction
Trying to debug an out of memory error with Postgresql.
Simple:
Open Connection
begin transaction
trying to import about 20GBytes of data (40K rows + rest large image blob's)
end transaction
...
0
votes
0answers
8 views
postgres encoding and django-actstream
I'm using django activity stream: https://github.com/justquick/django-activity-stream/
It seems to be working fine except that I created a follow-relationships for an object with the character é and ...
0
votes
2answers
28 views
denormalized numpy arrays from postgres
The following query pulls out ca. 100'000 datapoints into python. Data will be plotted with matplotlib.
cur.execute("""SELECT \
loggingdb_ips_integer.ipsvalue,
...
-2
votes
1answer
53 views
python convert unicode to readable character
I am using python 2.7 and psycopg2 for connecting to postgresql
I read a bunch of data from a source which has strings like 'Aéropostale'. I then store it in the database. However, in postgresql it ...
0
votes
1answer
11 views
how to auto-skip the repeated raw in executemany?
I set a primary key for my table. Now I want to auto-skip the repeated raw when I executemany Insert.
What should I do? Here's my code and comment. Thank you guys.
cmd = \
'''
CREATE TABLE test(
id ...
0
votes
0answers
21 views
sqlalchemy/postgresql - combining 2 queries into one using join
I have 2 sqlalchemy queries:
the first is a joined table between time-series and object_id (foreign key to other table):
q_time_series = session.query(
func.
generate_series(
...
0
votes
1answer
19 views
In postgres I see a table name which I didn't create at all
The name is strange as well, below is how the table name got created
<tablename like 'attribute value'> (I haven't given the original table name here, the whole string inside the angular ...
0
votes
2answers
20 views
Generating Boolean Expressions from Subqueries in SQLAlchemy
I have the following SQLAlchemy models:
PENDING_STATE = 'pending'
COMPLETE_STATE = 'success'
ERROR_STATE = 'error'
class Assessment(db.Model):
__tablename__ = 'assessments'
id = ...
0
votes
1answer
15 views
Unable to access tables created with Python program using sqlalchemy
I have created some tables using python script using sqlalchemy in the DB postgres.
when I log into db using shell for that uname:password combination and execute command to show tables
\dt
it gives ...
2
votes
2answers
27 views
Writing in SQLite multiple Threads in Python
I've got a sqlite3 database and I want to write in it from multiple threads. I've got multiple ideas but I'm not sure which I should implement.
create multiple connection, detect and waif if the DB ...
2
votes
0answers
15 views
How to format a SQLAlchemy ORM Postgresql CIDR contains (>>) query
I have a model defined with:
from app import db
from sqlalchemy.dialects import postgresql
class TableIpAddress(db.Model):
__tablename__ = 'ipaddress'
idipaddress = db.Column( postgresql.UUID, ...
0
votes
1answer
33 views
Best way to avoid data loss in a high-load Django app?
Imagine a quite complex Django application with both frontend and backend parts. Some users modify some data on the frontend part. Some scripts modify the same data periodically on the backend part.
...
0
votes
2answers
31 views
Can't access Postgres DB on Heroku app
I have a Python 2.7 / Flask app running on Heroku that scrapes data and I now want to store that information in a database.
I've tried to follow tutorials like this one and apply this to my case but ...
2
votes
2answers
38 views
How to avoid race condition with unique checks in Django
I have a simple model:
class InvitationRequest(models.Model):
email = models.EmailField(max_length=255, unique=True)
And a simple model form:
class InvitationRequestForm(forms.ModelForm):
...
0
votes
1answer
60 views
Why is django using sqlite3 when I want to use postgresql
I get this stupid error when I try to log into admin.
Exception Type:
OperationalError
Exception Value:
unable to open database file
Exception Location:
...
0
votes
0answers
22 views
postgres - remote connection - PgAdmin connects, php/python doesn't
I've got a VPN to my server network set up. Everything works fine, I can ping servers and ssh around the network. Except for the database connection.
When I use PgAdmin on ubuntu and insert ...
0
votes
0answers
12 views
django cursor excute sql does not return the right value
simple table:
id username gw_id lastlogin
1 180 1 2014-09-04 16:48:57+8
2 181 1 2014-09-04 16:48:04+8
the sql and use the navicat client exute the sql the result
SELECT
...
0
votes
2answers
25 views
Django: distinct on a foreign key, then ordering
I have two models, Track and Pair. Each Pair has a track1, track2 and popularity. I'm trying to get an ordered list by popularity (descending) of pairs, with no two pairs having the same track1. ...
1
vote
1answer
27 views
Getting Started with Python on Heroku - pg_config executable not found
I have been following the documentation until I got to the point of installing the requirements.txt file. It always fails when trying to install line 6 "psycopg2==2.5.3". Here is the message -
...
2
votes
1answer
78 views
Pull large amounts of data from a remote server, into a DataFrame
To give as much context as I can / is needed, I'm trying to pull some data stored on a remote postgres server (heroku) into a pandas DataFrame, using psycopg2 to connect.
I'm interested in two ...
0
votes
1answer
32 views
Python read and write 2 DBs
I have never really used python before.
Before getting into it, I had a quick question to check the feasibility of something I am planning to do here.
I have two servers, MySQL server and postgresql ...
1
vote
2answers
45 views
Manually escape a string for raw SQL
I have a raw SQL statement to be executed using SqlAlchemy. The query contains this:
FROM unnest(array['sku1', 'sku2', 'sku3'])
Building the query manually, I could not find a way to escape each ...
2
votes
1answer
31 views
Why does upgrading Django 1.7c2 to 1.7c3 create new database migrations?
This is not a problem so much as a curiosity, but I've looked around and can't find an answer to this.
When I upgraded Django 1.7c2 to 1.7c3 today, Django identified that I needed new migrations (ie ...
0
votes
1answer
53 views
Celerybeat schedule executing task multiple times?
I have a task calculate_common_locations which runs once via CELERYBEAT_SCHEDULE.
The task simply calls a function in the database:
@app.task
def calculate_common_locations():
db.execute("SELECT ...
0
votes
1answer
36 views
Copy (from) csv with heades in postgres with python
I'm trying to fill a table from CSV files in a python script.
The SQL statement, which follows, runs without error:
COPY registro
FROM '/home/pablo/Escritorio/puntos/20140227.csv'
DELIMITER ','
CSV ...
0
votes
0answers
33 views
Django operational error in postgresql
im having an issue with django, im trying to create a data base on django for my application i have set my conection like this
DATABASES = {
'default': {
'ENGINE': ...
1
vote
0answers
20 views
Django SQL backend strange output with timestamp/epoch
I don't understand what's going on and what to write to google to find solution:
Seems like Django PostgreSQL database engine doing something strange?
first - simple query:
>>> data = ...
0
votes
2answers
43 views
How to efficiently define daily constant?
Using the (awesome) Python Flask framework, I'm building a website, for which I have 3 "constants" which change every morning (so my constants are not very constant, I know, but still). I need these ...
1
vote
1answer
49 views
Creating a query with few related tables in Pyramid with SQLAlchemy
I have defined few tables in Pyramid like this:
# coding: utf-8
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Integer, Float, DateTime, ForeignKey, ...
0
votes
1answer
37 views
Error: `gyp` failed with exit code: 1 while installing jugglingdb-postgres
I am getting this error while installing jugglingdb-postgres in my ubuntu.help me to sort out this problem.Using Python 2.7.4 in my ubuntu.
gyp: Call to 'pg_config --libdir' returned exit status 1.
...
0
votes
1answer
15 views
How to write a SQLAlchemy event listener for when a view within a specific Postgres schema is dropped?
In my app, whenever a view within a certain Postgres schema is dropped, I want to call a specific function. I am familiar with SQLAlchemy's event.listener functionality, but I have mainly seen it used ...
0
votes
1answer
17 views
Postgresql ssl connection through python
I found below link
http://python.projects.pgfoundry.org/docs/1.0/driver.html#connection-keywords
it says...
import postgresql.driver as pg_driver
is the way to import
i used -
import ...
0
votes
1answer
15 views
Storing to database in sqlalchemy with no values
I am trying an api in sql alchemy in which i am storing details of courses into database. The below code is the one i am used in which course name, start date, end date and hours per day are from the ...
0
votes
2answers
43 views
How to Profile Python Database Code
I have a small Python project (~5000 lines) that runs certain heavy statistical calculations on medium-sized Postgres database tables (~50 million entries), using the pg library.
The nature of the ...
0
votes
1answer
14 views
SQLAlchemy One-To-One and One-To-Many at the same time (AmbiguousForeignKeysError)
I'm working with SQLAlchemy and I try to achieve one-to-one and one-to-many relationships on the same parent class.
This is for the simplicity of keeping track of the main child entity.
Unfortunately ...
1
vote
2answers
21 views
How to do division of cells in a csv file before importing into postgres using a python script?
I want to insert price of a product by doing sales dollars/sales units (or 0 if sales unit is 0) into a postgres sql. Is there I can do this when inserting into the database or do I have to run an ...
0
votes
1answer
30 views
Why is Django ManyToManyField causing admin interface to crash? Why is no through table being created?
Why is this line
users_favorited_by = models.ManyToManyField('auth.User')
or this one, which causes the same error:
users_favorited_by = models.ManyToManyField(User)
in this model
class ...
0
votes
0answers
39 views
Why am I gaining an hour when converting to/from epoch timestamp? [duplicate]
example = datetime.datetime(2014, 8, 19, 14, 18, 49, 435413, tzinfo=psycopg2.tz.FixedOffsetTimezone(offset=-240, name=None))
datetime.datetime.fromtimestamp(time.mktime(example.timetuple()))
...
0
votes
0answers
28 views
Passing a string with “accents” to psycopg
I'm trying to create a script that will upload everyday some files to our Postgres DB.
I create a python script that reads all the .csv files in a given directory:
# -*- coding: utf-8 -*-
import os
...
0
votes
1answer
30 views
psycopg2: Insert a numpy array of strings into an PostgreSQL table
Please consider me to be a complete novice with psycopg2. My aim is to insert a 1D numpy array of dtype object (where the elements are only strings) into a postgresQL table. My main program saves the ...
0
votes
1answer
47 views
Update PostgreSQL database with daily stock prices in Python
So I found a great script over at QuantState that had a great walk-through on setting up my own securities database and loading in historical pricing information. However, I'm not trying to modify the ...
0
votes
0answers
54 views
TransactionManagementError - This is forbidden when an 'atomic' block is active
While upgrading my project from django 1.5.1 to 1.6.5, I am facing this weird issue.
This is forbidden when an 'atomic' block is active.
I am aware of the Database Transaction changes for django ...
1
vote
1answer
53 views
For an SQLAlchemy query, how to combine ilike search operator with in_ operator?
I am writing SQLAlchemy code that takes a search bar's input and runs a query against my PostgreSQL database. To allow for misspellings and name fragments, I had this code that did the trick:
q = ...
0
votes
1answer
51 views
Limiting the subquery with SQLAlchemy
I have defined few models in SQLAlchemy:
class Location(Base):
__tablename__ = 'Location'
__table_args__ = {u'schema': 'Location'}
Unit_id = ...
1
vote
1answer
29 views
Postgresql hmac-sha1 signature differs from python signature
Postgres:
Code:
SELECT encode(
hmac(
E'PUT\n\n1\n1408355972\nx-amz-acl:bucket-owner-full-control\n/1/1',
'1sf235123',
'sha1'
),
'base64'
);
Result: ...
0
votes
1answer
38 views
Django-cities exits with “killed”
I am trying to install django-cities on a server with 500mb RAM. Whenever I run:
manage.py cities
the system exits just saying killed.
Trying to run:
manage.py cities --import=country ...
0
votes
0answers
37 views
Installing and using postgis on ubuntu
I have a site running on django and I'm adding the django-cities module to it.
So far I have installed postgresql-9.3-postgis-2.1 and made the necessary changes to the database by running the CREATE ...
0
votes
1answer
24 views
Commit() in psycopg2 seems to be not executed
And so I have this Python file (dosql.py):
import sys
import psycopg2
class doSql(object):
#attributes
_cxn = ""
_cur = ""
errmsg =""
#methods
def __init__(self): ...
0
votes
0answers
25 views
Using psycopg2 cursor.copy_from and StringIO with strings including newline characters
I've been trying to find a solution to the following problem without success.
I want to copy data (different types, floats, strings, etc.) using copy_from method of psycopg2 cursor class, to my db.
...
1
vote
2answers
96 views
using list on postgresql JSON type with sqlalchemy
I am using pyramid with sqlalchemy, pyramid_tm and postgresql to test this.
DBSession = scoped_session(sessionmaker(extension=ZopeTransactionExtension()))
Base = declarative_base()
class ...
